I've been working with Google Earth Network links for some time, and have learned a few things along the way. I am by no means an expert, but hopefully you can learn a little from my experiences. My experiences come from working on ActiveTrails.com, and you can see a running example of these methods there. My challenge was to make the ActiveTrails Network Link just as fast as the layers inside Google Earth. I accomplished my goal (close enough to satisfy me, anyway), and I talk about the key points here. First, some background about Google Earth and Network Links.

Google Earth

Google Earth is a fantastic program. Originally developed by Keyhole Systems, Google bought the company and continued to develop the program. Most remarkable is the ability to 'tilt' your view so that you can see mountains and terrain in 3D. Google earth has many layers that contain geographic data. Some display 3D buildings of major cities, or 3D terrain, while others display geotagged pictures or Wikipedia entries. Our goal is to emulate one of these layers, both in functionality and speed. Google Earth is a free download from http://earth.google.com

KML and Network Links

Google Earth has the ability to display data imported from outside sources. KML (Keyhole Markup Language) is an XML syntax used to create data files for use in Google Earth. It is fairly easy to create these files dynamically, allowing users to download a file created from data on your site and display it in Google Earth. If you visit a Trail page in ActiveTrails and click the Google Earth link under export, you can see this in action.

Downloadable KML files will allow your users to download the current version, or snapshot of your data. If you add or update information after a user downloads the file, their copy will be missing the recently updated information. A Network Link is like a live version of a KML file. It links Google Earth to your server, so that new information is downloaded fresh, showing all the recent updates.

View Based Refresh

Google Earth provides several ways for network links to interact with your server. The most common method is a view based refresh. When the view inside Google Earth stops for a set number of seconds, it makes a request to your server, sending you the latitude/longitude boundaries of the area currently in view. Your server can use that information to send back information for that particular geographic area.

This method works well for small datasets, or situations where you are only showing markers on the map. If the amount of data returned is too large, then this method will place a great deal of load on your server, and will also be very slow for the users of your network link.

Region Based Refresh

In the newest versions of Google Earth, they introduced the Region based refresh method. This allows data to be downloaded when a region is in view. This method is great, because the data is downloaded just once for each region. Subsequent displays are done from the cache, making the experience much faster for the user.

Regions can also contain sub-regions. This allows you to break up your data into smaller files that will be reconstructed into your full dataset when downloaded. Each region is downloaded when it has reached the minimum size specified, and is served out of the cache as well. This method allows you to send just the needed data to your users, but still make it fast and nice to use. The real advantage is that you do not need to know the user's current view to generate network link files, allowing you to pre-generate them.

KML + Zip = KMZ

XML files are not known for being lightweight, and KML files are no exception. Each trail on ActiveTrails has both a trailhead and a trail. Trailheads are marked with a pin, and only a single latitude/longitude pair is required. The trail is represented by a series of latitude, longitude, and elevation points, and this data can be quite large. Even after breaking up your data into regions, the files can still be large enough to eat significant bandwidth and slow down the experience for your users.

Google Earth also accepts KMZ files, which are compressed KML files. KMZ files are really just zip files with a different extension, which makes them very easy to work with. By creating the KML file you need, and then compressing it into a zip file and giving it the .kmz extension, you can reduce the download burden of the file. While XML itself can be bulky, it compresses well, and you will see significant bandwidth savings. Google Earth handles KMZ files exactly the same as KML files.

Note: If your KML files contain links to regions, you will need to have the link point to the .kmz file of it's subregion, not the .kml file.

Amazon S3

Amazon has a division called Amazon Web Services, where they host some pretty fascinating services. One of these is the Simple Storage Service, or S3 for short. I've heard S3 described as a 'big hard drive in the sky' and that pretty much sums it up. S3 is like a web server that can host data either publicly or privately, and is fast, distributed, and cheap.

ActiveTrails, like many small and growing websites, runs on just one server. If the site were to suddenly become much more popular, the server would surely groan and fall under the pressure of a million clicks. My solution to this is to not make my poor little server host the entire website. I can put pieces of it on S3, and link to it as if I were running a second server to host media. This is what I've done with my Network Link, and I'll explain why it's useful:

It's Cheap. Amazon charges for upload, storage, and download of data at pretty cheap rates. If I run over my included bandwidth at my current hosting provider, my rates are much higher. It is true that you can get cheap hosting by obtaining a basic hosting account for a few dollars a month and storing your data there, but that solution is not as simple, flexible, and scalable as using S3.

It Scales. Amazon describes their web services as being a key part of 'web-scale' computing. They currently serve media files for some pretty large customers, and I can get the same kind of service as the big players. S3 can easily handle any kind of traffic that I'm capable of throwing at it. Some have complained at the lack of a Service Level Agreement from Amazon. I figure that their worst performance is far better then my best.

It's fast. Amazon has designed S3 to be very fast with distributed data centers and fast pipes. The performance I experience from S3 is far better then I've ever seen on a regular basis from any normal website.

By hosting the KMZ files on S3 instead of my own server, my network link can now withstand any kind of traffic with fast response times. To change the location of hosted network link files, you must also change the links within those files so that they will link to each other on S3, instead of the files on your own server.

Automatic updates

Any worthwhile website has data that grows and changes over time. You will need to update the files that make up your network link, and there are some things to know.

You only need to update the files where data has changed. If you have broken your data into multiple files in the process of creating your network link, then you only need to update the files that have been touched by the update. This can save processing time and upload bandwidth.

Google Earth caches the files that users download, and will only re-download them if they have been updated. S3 does a great job with Etags, last modified headers, and Head requests for your data, and this saves some S3 bandwidth by only downloading new data. All this means that you simply update the network link files on S3, and your users will see new data the next time they launch Google Earth.

Conclusion

Well, there you have it. Wicked Fast Google Earth Network Links using KML, Zip files, and Amazon S3. I've been very pleased with the results, and I hope you are as well. The concepts here are somewhat specific to the requirements of ActiveTrails, so you may need to modify these concepts to apply them to different projects. Now go make a wicked fast network link of your own, and share it with the world.