(Back home)

Technology behind the chart

This describes the components used in the current Rosy.asp charting page (see related discussion in the installation guide). The system has evolved from its original MS Access-based backend to a MySQL database.

• Data-Collection Scripts
  • Perl: Some of the data behind the chart is gleaned (screen scraped) from web pages. Please refer to the FAQ on data sources. This includes the HMS web page and the two NOAA web pages. A Perl script runs every 5 minutes and collects all the text from these pages and then parses out wind, temperature, and pressure data. Perl has useful features for manipulating text data. This data is inserted into a MySQL database on the server. The charting application then gets its data from the database.
  • Python: The MN data added in 2006 uses a scheduled Python script to download and parse METAR data from an XML data feed. Additional Python scripts handle data from other sources including mesoscale stations and airport weather data, all feeding into the central MySQL database.
• Database Connectivity
  • ADO (ActiveX Data Objects): The ASP applications create ADO Connection objects to establish database connectivity. The actual connection implementation uses:
    • System DSN Configuration: Database connections use a pre-configured system DSN named "MySQL_telem_for_ASP" which encapsulates the MySQL server connection details and credentials.
    • MySQL ODBC 5.3 Unicode Driver: The DSN utilizes the 32-bit MySQL ODBC Unicode driver to provide the underlying database connectivity layer.
    • Telem Schema: All database operations target the "telem" schema which contains the weather station data tables.
• MySQL Database: The Perl and Python scripts populate a MySQL database on the server, replacing the original MS Access database. The database stores the HMS data in a resolution of 15-minutes and the NOAA data in a resolution of 1 hour, plus additional mesoscale and airport station data. One of the charting options, delta-P, makes use of SQL queries that serve to join the pressure records from different sites and calculate their differential pressures. MySQL provides improved performance, concurrency, and data integrity compared to the original Access database.
• Charting Scripts
  • Server-Sided
    • ASP (Active Server Pages): This is what makes the dynamic chart possible. It collects the form parameters and runs queries on the MySQL database.  It then creates a chart image (JPG file), using ASPChart. The ASP code then acts to write the HTML code (for the web page), which includes a needed reference to the image file.
    • VBScript (Visual Basic Scripting Edition): The ASP code is primarily written in VBScript which is a subset of VBA (Visual Basic for Applications).
    • ASP Chart: This is a freeware charting component for ASP. It produces the chart image.
    • JavaScript: A portion of the server sided ASP code is written in Javascript.
  • Client-Sided
    • JavaScript: Client code that runs in the browser is written in JavaScript. An example of this is the code that supports the timer function.
• Editors
  • Microsoft Expression Web: Editor for laying out static HTML pages.
  • Notepad++ and VIM: Editors for the code.
• Operating System: It all runs on MS Windows 7, 8, 10, and 11.
  • IIS: Internet Information Server. This is Microsoft's web server and executes the server-sided code of the charting page (ASP).
  • MS Window's Scheduling Service: The Perl and Python gleaner scripts are scheduled to run every five minutes throughout the day.