The Australian Bureau of Meteorology public FTP server, part 0: overview

Here at Ice Moon Prison, we’ve been talking about the weather for years. We’ve always used the free data supplied by the Bureau of Meteorology. This is the first in a series of posts outlining what data can be found on the BOM’s FTP server.

The Bureau of Meteorology is an Australian federal government agency, supplying Australian individuals and organizations with observations, forecasts, and warnings. Mostly for a fee, which runs to 4 to 5 figures depending on the data being used, but a subset of the data can be had for free, with strings attached.

The data is on the server ftp.bom.gov.au and can be accessed over the venerable anonymous File Transfer Protocol. In years past, that link would open in your browser, but as anonymous FTP gave way to HTTP, browser support for FTP was dropped, so you may have to experiment with other options to access the files.

The BOM’s own documentation on the anonymous FTP data is sparse and inconsistent. You could spend fruitless hours exploring the uncharted, cryptic directories. I certainly did. So I’ve decided to write these blog posts and save you time getting productive.

Here’s what you’ll need:

• An FTP client. Windows and MacOS have clients built into their file browser (File Explorer and Finder, respectively). You can also use a dedicated client like FileZilla. If you’re interested in automating or scripting, you’re almost certainly going to want to use curl.
• A shapefile viewer. This is optional, but a map overlay display tool like QGIS will help you to visualize which BOM products apply to a particular part of Australia. If you skip this, you can interactively explore the BOM’s website to find which products have the information you need.
• An XML processor. Most of the BOM’s textual products come in valid XML, and if you want to manipulate these, you’d best come to terms with a processor like Saxon-HE or libxml2. XSLT, XPath, and to a lesser extent SAX parsing, will make this a breeze. Please don’t hurt yourself by trying to parse XML with regular expressions.

Future posts in this series will tentatively cover the following topics:

1. Finding yourself: identifiers, codes such as AMOC and AAC, shapefiles and the spatial folder.
2. Know your product: how the BOM packages things, what a product is, and the radar folder.
3. Predicting the future: forecasts, parsing XML, finding the right product, and the fwo folder.
4. What did you see?: observations and synoptic charts.
5. Proceed with care: finding and processing warnings.