Comments
yourfanat wrote: I am using another tool for Oracle developers - dbForge Studio for Oracle. This IDE has lots of usefull features, among them: oracle designer, code competion and formatter, query builder, debugger, profiler, erxport/import, reports and many others. The latest version supports Oracle 12C. More information here.
Cloud Expo on Google News
SYS-CON.TV
Cloud Expo & Virtualization 2009 East
PLATINUM SPONSORS:
IBM
Smarter Business Solutions Through Dynamic Infrastructure
IBM
Smarter Insights: How the CIO Becomes a Hero Again
Microsoft
Windows Azure
GOLD SPONSORS:
Appsense
Why VDI?
CA
Maximizing the Business Value of Virtualization in Enterprise and Cloud Computing Environments
ExactTarget
Messaging in the Cloud - Email, SMS and Voice
Freedom OSS
Stairway to the Cloud
Sun
Sun's Incubation Platform: Helping Startups Serve the Enterprise
POWER PANELS:
Cloud Computing & Enterprise IT: Cost & Operational Benefits
How and Why is a Flexible IT Infrastructure the Key To the Future?
Click For 2008 West
Event Webcasts
Amazon’s Elastic Block Store Opens Up S3 and The Cloud
The Big SAN in the Sky

With EBS volumes up to 1TB in size and beyond in a file system is possible to mount quickly – and multiple volumes on the same instance, such that file systems of 10TB are practical. The volumes can further be backed-up to S3 using snapshots, and they can be replicated by creating new volumes from the snapshots. What is particularly nice is that a volume can be created in any availability zone (think datacenter) of a region from a snapshot, so copying a large volume across datacenters can be off-loaded to EBS and is done very efficiently.

Incremental snapshotting of volumes and freezing
I mentioned that snapshots were a very useful but also a complex, difficult to understand feature. I wanted to explain how snapshots of an EBS volume can be taken at any time, and why using RightScale’s scripts to freeze data are important -- and a great add-on to this important feature.

Taking a snapshot causes the data on the volume to be written to S3 where it is stored redundantly in multiple availability zones as all data in S3 is. It’s worth noting snapshots do not appear in your S3 buckets, thus you can't access them using the standard S3 API. You can only list the snapshots using the EC2 API and you can restore a snapshot by creating a new volume from it.

The second thing is that snapshots are incremental, which means that in order to create a snapshot, EBS saves only the disk blocks that have changed to S3.

Each volume is divided up into blocks. When the first snapshot of a volume is taken, all blocks of the volume that have ever been written are copied to S3, and then a snapshot table of contents is written to S3 that lists all these blocks. Now, when the second snapshot is taken of the same volume, only the blocks that have changed since the first snapshot are copied to S3. The table of contents for the second snapshot is then written to S3 and lists all the blocks on S3 that belong to the snapshot. Some are shared with the first snapshot, some are new. The third snapshot is created similarly and can contain blocks copied to S3 for the first, second and third snapshots.

There are two nice things about the incremental nature of the snapshots: it saves time and space. Taking subsequent snapshots can be very fast because only changed blocks need to be sent to S3, and it saves time because you're only paying for the storage in S3 of the incremental blocks. What is difficult to answer is how much space a snapshot uses. Or, to put it differently, how much space would be saved if a snapshot were deleted. If you delete a snapshot, only the blocks that are only used by that snapshot (i.e. are only referenced by that snapshot's table of contents) are deleted.

Something to be very careful about with snapshots is consistency. A snapshot is taken at a precise moment in time even though the blocks may trickle out to S3 over many minutes. But in most situations you will really want to control what's on disk vs. what's in-flight at the moment of the snapshot. This is particularly important when using a database. We recommend you freeze the database (or any application writing critical data to disk), freeze the file system, take the snapshot, then unfreeze everything. At the file system level we've been using xfs for all the large local drives and EBS volumes because it's fast to format and supports freezing. Thus when taking a snapshot we perform an xfs freeze, take the snapshot, and unfreeze. All this ensures that the snapshot doesn't contain partial updates that need to be recovered when the snapshot is mounted.

With support for large datasets, attachments, better throughput, snapshotting and more robust, incremental backups and redundancy, Amazon’s EBS should attract a lot more enterprise and on-demand customers, as well as Web 2.0 users with large database-driven applications.

Thorsten von Eicken is RightScale, Inc.’s Chief Technical Officer. To try out a free developer version of RightScale, visit http://www.rightscale.com/m/products.html#developer.

About Thorsten von Eicken
Thorsten von Eicken is the CTO and a founder of RightScale and is responsible for the overall technology direction of the RightScale Cloud Management Platform. Previously, he was founder and chief architect at Expertcity (acquired by Citrix Online), where he directed the architecture of the company's online services, including the popular GoToMeeting service. von Eicken also managed the Expertcity/Citrix data center operations, where he acquired deep knowledge in deploying and running secure, scalable online services. He was a professor of Computer Science at Cornell University and received his Ph.D. from the University of California, Berkeley.

In order to post a comment you need to be registered and logged in.

Register | Sign-in

Reader Feedback: Page 1 of 1

Dr von Eicken will be giving a technical session at SYS-CON's "Cloud Computing Expo" (November 19-21, 2008) - a major adjunct to the 4th International Virtualization Conference & Expo being held at The Fairmont Hotel in San Jose, CA - in which he will distill the unique characteristics of clouds and describe how to best think about deployments in the clouds.


Your Feedback
Jeremy Geelan wrote: Dr von Eicken will be giving a technical session at SYS-CON's "Cloud Computing Expo" (November 19-21, 2008) - a major adjunct to the 4th International Virtualization Conference & Expo being held at The Fairmont Hotel in San Jose, CA - in which he will distill the unique characteristics of clouds and describe how to best think about deployments in the clouds.
Latest Cloud Developer Stories
Having been in the web hosting industry since 2002, dhosting has gained a great deal of experience while working on a wide range of projects. This experience has enabled the company to develop our amazing new product, which they are now excited to present! Among dHosting's greate...
Nutanix has been named "Platinum Sponsor" of CloudEXPO | DevOpsSUMMIT | DXWorldEXPO New York, which will take place November 12-13, 2018 in New York City. Nutanix makes infrastructure invisible, elevating IT to focus on the applications and services that power their business. The...
Concerns about security, downtime and latency, budgets, and general unfamiliarity with cloud technologies continue to create hesitation for many organizations that truly need to be developing a cloud strategy. Hybrid cloud solutions are helping to elevate those concerns by enabli...
Digital transformation is about embracing digital technologies into a company's culture to better connect with its customers, automate processes, create better tools, enter new markets, etc. Such a transformation requires continuous orchestration across teams and an environment b...
The complexity of managing and delivering the high level of reliability expected of web-based, cloud hosted systems today, and the expectation of Continuous Delivery of new features has led to the evolution of a totally new field of Service Reliability Engineering catered for suc...
Subscribe to the World's Most Powerful Newsletters
Subscribe to Our Rss Feeds & Get Your SYS-CON News Live!
Click to Add our RSS Feeds to the Service of Your Choice:
Google Reader or Homepage Add to My Yahoo! Subscribe with Bloglines Subscribe in NewsGator Online
myFeedster Add to My AOL Subscribe in Rojo Add 'Hugg' to Newsburst from CNET News.com Kinja Digest View Additional SYS-CON Feeds
Publish Your Article! Please send it to editorial(at)sys-con.com!

Advertise on this site! Contact advertising(at)sys-con.com! 201 802-3021



SYS-CON Featured Whitepapers
Most Read This Week
ADS BY GOOGLE