Make this Notebook Trusted to load map: File -> Trust Notebook
K-12 & Undergraduate STEM Outreach: NOAA Big Data Climate Challenge
Analyzing NYC Central Park Meteorological Trends with Python, GIS, and Data Structures
1 Mission Briefing: The NOAA Climate Intern
NoteOutreach Program Context
This module is tailored for middle school, high school, and introductory undergraduate STEM workshops. Students step into the shoes of a data science intern at the National Oceanic and Atmospheric Administration (NOAA) National Centers for Environmental Information (NCEI).
You have just joined the NOAA scientific team responsible for monitoring urban heat island effects across the New York metropolitan area. Your mentor assigns you to examine meteorological records from the historic Central Park Observatory (GHCND:USW00094728).
1.0.1 The Scenario
An urgent environmental policy meeting will convene this Friday with city planners. They require instantaneous access to daily high temperatures (T_{MAX}), low temperatures (T_{MIN}), precipitation (PRCP), and true daily mean temperatures (\bar{T}) between January 1, 2017 and June 2, 2017.
The challenge: The specific test dates will not be revealed until the meeting starts.
You cannot open a massive spreadsheet and click around while senior scientists wait. You must design and deploy a rapid, automated Python data pipeline that reads the raw scientific records and retrieves precise observations instantaneously.
2 Procuring the Raw Data
Before writing any code, data scientists must source high-quality, verifiable data. For this project, you will use the official public records from the U.S. Government.
2.0.1 Where to Download centralpark.txt
If your instructor has not provided the centralpark.txt file, you can pull it directly from the NOAA database yourself:
- Visit the NOAA NCEI Portal: Go to the Climate Data Online (CDO) Search tool.
- Configure Your Search:
- Dataset: Select Daily Summaries.
- Date Range: Select January 1, 2017 to June 2, 2017.
- Search For: Select Stations.
- Search Term: Type Central Park, NY.
- Add to Cart: Locate
NY CITY CENTRAL PARK, NY US(Station ID: GHCND:USW00094728) and click “Add to Cart”. - Checkout Options: Go to your cart. Select Custom GHCN-Daily Text (to get the
.txtspace-delimited format) or Custom GHCN-Daily CSV (for comma-delimited). - Download: Enter your email. NOAA will send you a link to download your custom dataset within minutes. Save this file in the same folder as your Python script and rename it to
centralpark.txt.
3 Geographic Context: Mapping the Observatory
The Central Park observation station has recorded weather continuously since 1869. Situated atop Belvedere Castle and Vista Rock, it represents one of the longest continuous climate records in North America.
3.0.1 Interactive Geospatial Station Map
The interactive GIS map below pinpoints the observation station inside Central Park relative to Manhattan’s urban canopy. Use the layer control in the top-right corner to toggle between standard street maps and Esri satellite imagery.
3.1 Build Your Own Interactive Map (For Students)
You can recreate the map above for any weather station or location in the world using Python’s folium library.
Step 1: Install the mapping library. Open your terminal or command prompt and run:
pip install foliumStep 2: Find your coordinates. Go to Google Maps, right-click any location (like your local high school or hometown weather station), and copy the Latitude and Longitude numbers (e.g., 40.869, -73.042 for Selden, NY).
Step 3: Write the mapping script. Create a new Python file (e.g., make_map.py) and paste the following code. This code utilizes free ArcGIS satellite tiles from Esri:
import folium
# 1. Define your target coordinates
my_lat = 40.77898
my_lon = -73.96925
# 2. Create the base map
my_map = folium.Map(location=[my_lat, my_lon], zoom_start=16)
# 3. Add high-resolution ArcGIS Satellite Imagery
folium.TileLayer(
tiles="https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{z}/{y}/{x}",
attr="Tiles © Esri",
name="ArcGIS Satellite"
).add_to(my_map)
# 4. Add a customized pin/marker
folium.Marker(
location=[my_lat, my_lon],
popup="<b>My Weather Station</b>",
tooltip="Click Me!",
icon=folium.Icon(color="blue", icon="info-sign")
).add_to(my_map)
# 5. Add a layer control toggle and save to an HTML file
folium.LayerControl().add_to(my_map)
my_map.save("my_station_map.html")
print("Map generated successfully! Open my_station_map.html in your browser.")Run this code, open the resulting HTML file in Chrome or Safari, and you will have your own interactive GIS map!
4 Data Exploration & Time Series Visualization
Below is the dynamic historical time series of temperatures recorded at Central Park across the target study window. Hover over the lines to see exact daily temperatures.
4.1 Build Your Own Interactive Plot (For Students)
Data visualization is a core skill for any engineer. You can build the interactive chart above using pandas and plotly.
Step 1: Install the libraries.
pip install pandas plotlyStep 2: Write the plotting script. If you have successfully completed the core lab and loaded your data into a dictionary (or a Pandas DataFrame), you can graph it using Plotly Graph Objects (go). Create a script named plot_weather.py:
import pandas as pd
import plotly.graph_objects as go
# 1. Assuming you parsed the centralpark.txt file into a DataFrame
# (For practice, here is a tiny fake dataset to test the plot)
data = {
"Date": ["2017-01-11", "2017-01-12", "2017-01-13"],
"TMAX": [52, 66, 62],
"TMIN": [42, 47, 32]
}
df = pd.DataFrame(data)
# 2. Calculate the Daily Average
df["TAVG"] = (df["TMAX"] + df["TMIN"]) / 2
# 3. Initialize the Figure
fig = go.Figure()
# 4. Add the Daily High Line (Red)
fig.add_trace(go.Scatter(
x=df["Date"], y=df["TMAX"],
name="High Temp", line=dict(color="red"), mode="lines+markers"
))
# 5. Add the Daily Low Line (Blue)
fig.add_trace(go.Scatter(
x=df["Date"], y=df["TMIN"],
name="Low Temp", line=dict(color="blue"), mode="lines+markers"
))
# 6. Make it look professional
fig.update_layout(
title="My Weather Station Data",
xaxis_title="Date",
yaxis_title="Temperature (°F)",
template="plotly_white"
)
# 7. Generate and open the interactive graph
fig.show()5 Computer Science & Data Architecture Concepts
5.1 The File Structure & The Sentinel Value Problem
A sample record from the NOAA flat file (centralpark.txt) is formatted as follows:
GHCND:USW00094728 NY CITY CENTRAL PARK NY US 20170111 0.52 0 0 -9999 52 42Notice that column 7 (TAVG) contains -9999.
WarningScientific Data Trap: Sentinel Values
In observational physical science, sensors occasionally drop packets or fail to record a parameter. NOAA encodes missing data with a numerical sentinel value like -9999. If a novice programmer calculates the seasonal average temperature using this column directly, the result will be wildly corrupted! You must defensively compute the daily average yourself:
\bar{T} = \frac{T_{MAX} + T_{MIN}}{2}
5.2 Lists ([]) vs. Dictionaries ({}): Algorithmic Time Complexity
Why should we transform this file into a Python dictionary instead of scanning through a list?
| Data Structure | Search Complexity | Analogy |
|---|---|---|
List [] |
\mathcal{O}(n) (Linear Time) | Flipping through a notebook page-by-page until you find the date |
Dictionary {} |
\mathcal{O}(1) (Constant Time) | Using an index/hash table to flip directly to the exact page |
If an executive asks for 100 dates during a meeting: * The List Approach: Scans the text file 100 separate times (\mathcal{O}(100 \cdot n)). * The Dictionary Approach: Reads the file once at startup. Each date query is answered instantly (\mathcal{O}(1)).
6 Core Lab: The Rapid Query Engine
6.1 Step 1: Pre-Lab Algorithm Design
In your engineering notebook, outline the 5 to 7 logical steps your code must execute to achieve the goals of the meeting: 1. Specify the path to centralpark.txt. 2. Open and read each line of the file. 3. Parse and split each line into distinct tokens. 4. Convert text strings into floating-point numbers. 5. Populate a dictionary keyed by date string ('YYYYMMDD'). 6. Prompt the user for input and retrieve results in constant time.
6.2 Step 2: The Complete Reference Solution
Below is the clean, production-grade implementation that avoids all common loop-leakage and parsing traps.
# ==============================================================================
# NOAA Central Park Climate Query Engine
# Author: Wesley Francillon, PhD
# ==============================================================================
def load_climate_database(filepath: str) -> dict:
"""
Parses space-delimited NOAA weather station data into an O(1) lookup dictionary.
"""
weather_db = {}
with open(filepath, 'r') as file:
for line in file:
tokens = line.strip().split()
# Skip empty lines and header rows
if not tokens or tokens[0] in ("STATION", "STATION_NAME"):
continue
# Verify row length matches Central Park schema
if len(tokens) >= 14:
date_key = tokens[7]
try:
prcp = float(tokens[8])
snow = float(tokens[10])
tmax = float(tokens[12])
tmin = float(tokens[13])
# Compute genuine mean temperature, bypassing sentinel -9999
tavg = (tmax + tmin) / 2.0
weather_db[date_key] = {
"TMAX": tmax,
"TMIN": tmin,
"TAVG": tavg,
"PRCP": prcp,
"SNOW": snow
}
except ValueError:
# Skip malformed numerical records
continue
return weather_db
def query_engine(db: dict):
"""
Interactive prompt allowing users to query dates with zero search loops.
"""
print(f"Database online. Loaded {len(db)} observation records.")
print("Type 'exit' to quit.\n")
while True:
target_date = input("Enter observation date (YYYYMMDD): ").strip()
if target_date.lower() in ('exit', 'quit', 'q'):
print("Session ended.")
break
# O(1) Constant-Time Lookup (No loops!)
record = db.get(target_date)
if record:
formatted_date = f"{target_date[:4]}-{target_date[4:6]}-{target_date[6:]}"
print("\n" + "=" * 42)
print(f" NOAA Central Park Summary: {formatted_date}")
print("=" * 42)
print(f" High Temperature (TMAX) : {record['TMAX']:>6.1f} °F")
print(f" Low Temperature (TMIN) : {record['TMIN']:>6.1f} °F")
print(f" Calculated Average : {record['TAVG']:>6.1f} °F")
print(f" Precipitation : {record['PRCP']:>6.2f} in")
print(f" Snowfall : {record['SNOW']:>6.2f} in")
print("=" * 42 + "\n")
else:
print(f"[!] Date '{target_date}' not found. Please enter a valid date (e.g., 20170111).\n")
# To run the system locally, uncomment the lines below:
# db = load_climate_database("centralpark.txt")
# query_engine(db)7 Student Discussion & Inquiry Questions
- Urban Heat Island Effect: Compare the Central Park temperature readings with suburban Long Island stations on the same date. Why does Manhattan frequently maintain higher nighttime low temperatures (T_{MIN})?
- Extreme Weather Events: On March 14, 2017 (Stella Blizzard), Central Park recorded 7.6\text{ inches} of snowfall and 1.97\text{ inches} of liquid water equivalent. Query this date in your program. How cold did it get?
- Data Integrity: What would happen if your code checked
if date in row:rather than parsing individual columns? (Hint: Consider what happens when searching for the number12).