Greenplum, the high-end open source database house and friend of Sun – so it’s gotta be nibbling on its nails wondering whether it’s lost a channel – has some sexy new technology to accelerate data loading for companies stressing under exponential data growth.

It’s called MPP Scatter/Gather Streaming or SG Streaming for short and it’s supposed to eliminate those darn bottlenecks usually associated with mainstream data loading.

Greenplum claims to have created the lightening-fast flow of data into its database for large-scale analytics and data warehousing. It says it’s getting loading speeds of over 4TB an hour in production with a negligible impact on concurrent database operations.

Let’s pause for a moment to consider that one terabyte is equal to a goose bump-provoking million books and that Greenplum’s claimed rates move the industry closer to real-time data warehousing.

Greenplum uses a shower head-like parallel-everywhere approach to loading in which the data flows from hundreds or thousands of parallel streams to every node of the database without any sequential choke points, quite different from the usual drip-drip-drip bulk-loading single-source stream like Oracle.

And the widgetry scales. The more nodes, the faster the loading rate so it can theoretically support better than 4TB an hour.

Data can be transformed and processed in-flight for extremely high-performance ELT (extract-load-transform) and ETLT (extract-transform-load-transform) loading pipelines.

The company says its precedent-setting approach avoids the need for a “loader” tier of servers (think MPP) that adds complexity and cost.

The widgetry supports both large batch and continuous near-real-time loading. Final gathering and storage of data takes place on all nodes simultaneously with data – compression is an option – automatically partitioned across nodes.

Greenplum says other parallel databases are as limited as traditional databases. Netezza, for instance, forces data to enter the system via a single node.