Handling time zones with Python. This post demonstrates the… | by Himalaya Bir Shrestha | Apr, 2023

By Jessie Hobb On Apr 21, 2023

This post demonstrates the functionalities to deal with time zones in Python by means of comparative assessment of hourly solar irradiance data for four cities in 2020 based on different time zones.

When I start my work in Bonn, Germany on the first day of October at 9 am, it is already afternoon at 12:45 pm in my hometown in Chitwan, Nepal. My friend in Sydney, Australia has already finished his work schedule at 6 pm on the same day. Another friend in New York, the USA is still sleeping as it is 3 am in morning there. This implies that these four places have different time zones.

Illustration of different time zones in the world. Map © OpenStreetMap contributors. Labels added by Author.

The time zone is an area, which observes uniform standard time for legal, social, or commercial purposes. The world is not uniformly divided into different time zones based on longitudes. Time zones tend to rather follow boundaries between and within countries for differentiation.

All time zones are defined as an offset from Coordinated Universal Time (UTC). And these values can range from UTC-12:00 to UTC+14:00. While the offsets are usually a whole number of hours, a few zones are also offset by an additional 30 or 45 minutes. For example, the time zone of Nepal has a time offset of UTC+05:45. In total, there are 38 time zones in the world.

If I have data on solar irradiance for the four cities in Nepal, Germany, Australia, and the USA in the UTC time zone, it doesn’t reflect the data for the same hour of the day in each of these countries. In this post, I am going to discuss how the time zones of the data can be handled for datetime objects including pandas dataframe in Python.

For this purpose, I am going to download solar irradiance data for 2020 of these four cities/countries, compare and analyze the data when:

The data of each country is in the UTC time zone and
The data refers to the respective time zone of the country.

Let’s get started.

Geocoding to retrieve the coordinates of four cities

In the first step, I retrieve the coordinates of the four cities in four countries because I need them to extract the solar irradiance data. The process of extracting the geographical coordinates by providing the name of the place is called geocoding.

As shown below, I wrote a function for geocoding using the geopy package. The function utilizes Nominatim, which is an open-source service for geocoding that uses OpenStreetMap data to find locations on the earth by name and address.

from geopy.geocoders import Nominatim
geolocator = Nominatim(user_agent="app")def get_coordinates(place):
"""Return the latitude and longitude of the place."""
place_details = geolocator.geocode(place)
coordinates = (place_details[1][0], place_details[1][1])
return coordinates

I used the function to extract the coordinates of individual cities and create a pandas dataframe out of it as depicted in the screenshot below.

Dataframe is created containing latitude and longitude values of the four cities. Illustration by Author.

Accessing data using NASA Power API

The Applications Programming Interface (API) service of NASA Power allows to retrieve Analysis Ready Data. For this post, I download the solar irradiance data for four cities in hourly resolution. The parameter I use is All Sky Surface Shortwave Downward Irradiance (ALLSKY_SFC_SW_DWN) for 2020, which is described in more detail in the section below.

The data is called in UTC time zone format, although the hourly API also allows calling the data in Local Solar Time (LST) format by default.

The base_url configuration looks as follows:

base_url = r”https://power.larc.nasa.gov/api/temporal/hourly/point?parameters=ALLSKY_SFC_SW_DWN&community=RE&time-standard=UTC&longitude={longitude}&latitude={latitude}&format=JSON&start=2020&end=2020"

Next, I loop through the longitude and latitude of each place defined by geocoding in a list called places and request the hourly solar irradiance data for 2020. The full code for this step is given in the GitHub gist below:

Parameter description

The solar irradiance data refers to the total power (direct + diffused) obtained from the sun per unit area per hour (Wh/m²) on a horizontal plane at the surface of the earth under all sky conditions.

This parameter, also referred to as Global Horizontal Irradiance (GHI), is relevant to calculate the size of solar PV module needed to meet the given electricity demand as given in the formula below:

Equation 1 and 2 refer to formula for calculating the sizes of solar PV module and battery to meet given electricity demand based on solar irradiance and other technical parameters. Illustration by Author.

Basic statistics of given data

Global Horizontal Irradiance data downloaded for four cities. Illustration by Author.

The downloaded data is depicted in the plot above. The data shows higher solar irradiance in Sydney towards the beginning and end of the year, and lower towards the middle of the year. This pattern is opposite in the other three cities, which can be explained by the location of Sydney in the Southern hemisphere and other cities in the Northern hemisphere of the globe.

Statistics from the downloaded data. Illustration by Author.

It is observed that Chitwan, Nepal received the highest annual solar irradiance (1669 kWh/m²) in 2020 followed by Sydney, Australia (1631 kWh/m²), New York, the USA (1462 kWh/m²), and Bonn, Germany received the least (1193 kWh/m²).

However, the maximum solar irradiance received at a particular hour is highest for Sydney (1061.3 W/m²) followed by Chitwan (997 W/m²).

The minimum solar irradiance and the 25th percentile values for each city is zero because there is no solar irradiance during night hours.

Time zone Handling

1. Default pandas dataframe without “datetime” format index

As 2020 was a leap year, there were 366 days and as a result, the data was obtained for 8784 hours.

When the data is first downloaded, its index is of integer (int64) type as shown below:

Data when first downloaded from NASA Power website. Illustration by Author.

2. Converting integer type index to “naive” datetime index

The dataframe index can be converted into datetime type using pd.to_datetime() and specifying the format %Y%m%d%H for year, month, day and hours respectively.

Converting integer index to datetime index. Illustration by Author.

This change is also reflected when the dataframe is plotted as the months Jan to Dec of 2020 are visible in xticks as shown below:

Plotting pandas dataframe with “naive” datetime index shows months from Jan to Dec 2020 in the xaxis. Illustration by Author.

Although this dataframe has a datetime index, it does not have any information about time zones and daylight saving. Hence, the dataframe index is a naive datetime object. This is evident by checking the time zone info of one of the index of the pandas dataframe.

Checking the time zone info of the first datetime index of dataframe. Illustration by Author.

3. Localizing “naive” datetime object to “time zone aware” datetime object

The datetime module of Python can be used to access, retrieve and manipulate the date and time information.

By default, the datetime.now() function returns the current “local” date and time information. However, it doesn’t have any time zone and daylight saving information as time_now.tzinfo returns None in the code snippet below, implying it is a naive datetime object.

As of now (21 April 2023), I am in Nepal. Therefore, I localize the current time to “Asia/Kathmandu” time zone using the timezone.localize() module of pytz package. Now, the time_in_nepal is a time zone aware datetime object.

To get the current local time in Germany, I can use time_in_nepal.astimezone(timezone("Europe/Berlin")), which is also a time zone aware datetime object.

First cell shows the current local time time_now as a naive datetime object without time zone information. Second cell shows localizing time_now to Nepali time zone. Third cell shows converting Nepali local time to German local time. Illustration by Author.

4. Localizing timezone of pandas dataframe

Next, I localize the naive index of pandas dataframe to UTC time zone using df.tz_localize(tz = "UTC") as shown in the code screenshot below.

Localizing naive dataframe index to UTC time zone. Illustration by Author.

It is observed that the index of df is converted from naive index to time zone aware index of UTC time zone as shown above.

5. List of all possible time zone addresses

The list of all possible time zone addresses that can be referred are available using all_timezones module of pytz package. There are 594 such addresses. Some addresses can refer to same time zone. For example, Europe/Berlin, Europe/Amsterdam, Europe/Copenhagen all refer to same time zone.

List of possible addresses that can be referred to for time zones using pytz module. Illustration by Author.

6. Create new dataframe for each city and convert UTC time zone to corresponding local time zone

df contains the solar irradiance data of the four cities in UTC time zone. In this step, I create four dataframes out of each column of df. And then I convert the time zone of new dataframe from UTC to the local time zone of each city or country it belongs to. For example, the time zone of df_chitwan is converted using

df_chitwan.tz_convert(tz = "Asia/Kathmandu").

Creating different dataframes out of each column of df. The time zones of new dataframes are converted from UTC to respective time zone of the country/city.

It is to be noted that for countries which have daylight savings, this is automatically accounted for in the time zone conversion. For example, Nepal time is consistent with UTC + 05:45 throughout the year. However, for Sydney, Python automatically deals with daylight saving as the offset with UTC time zone can be 10 or 11 hours depending on time of year.

7. Comparing the plots of solar irradiance data in different time zones

In this final step, I wanted to compare how the solar irradiance looked like in the four cities when the data corresponded to:

a. The UTC time zone and

b. The local time zone of each city.

In the code snippet below, I create two sub-plots to plot the solar irradiance in four cities. In the left subplot, the solar irradiance data for October 1, 2020 based on UTC time zone is plotted. And in the right subplot, the solar irradiance data for October 1, 2020 based on the local time of each city is plotted.

fig, (ax1, ax2) = plt.subplots(1, 2, figsize = (20, 6))
fig.suptitle("Solar irradiance on October 1, 2020")ax1.plot(df.loc["2020–10–01"])
ax1.set_title("Based on UTC time zone")
ax1.xaxis.set_ticks(ticks = df.loc["2020–10–01"].index[::4], labels = np.arange(0, 24, 4))
cities = df.columns.tolist()
handles = ax1.get_legend_handles_labels()[0]
ax1.legend(handles, labels = cities, loc = "upper right")
ax1.set_xlabel("Hour of day")
ax1.set_ylabel("W/m$^2$")
ax2.plot(df_chitwan.loc["2020–10–01"].values.tolist())
ax2.plot(df_newyork.loc["2020–10–01"].values.tolist())
ax2.plot(df_bonn.loc["2020–10–01"].values.tolist())
ax2.plot(df_sydney.loc["2020–10–01"].values.tolist())
ax2.xaxis.set_ticks(ticks = np.arange(0, 24, 4), labels = np.arange(0, 24, 4))
handles = ax2.get_legend_handles_labels()[0]
ax2.legend(handles, labels = cities)
ax2.set_title("Based on local time zone of each city/country")
ax2.set_xlabel("Hour of day")
ax2.set_ylabel("W/m$^2$")
plt.savefig("output/solar irradiance on october 1.jpeg",
dpi = 300)
plt.show()

The plot looks as shown below:

Solar irradiance on October 1, 2020. Left: Based on UTC time zone. Right: Based on local time zone of each city/country. Illustration by Author.

As of October 1, 2020, the time zones of four cities as compared to UTC time zone are: Chitwan (UTC+05:45), New York (UTC- 04:00), Bonn (UTC + 02:00), and Sydney (UTC+10:00). Thus, we see the solar irradiance peak around 4 am, 3 pm, 10 am and 3 am of UTC time zone for Chitwan, New York, Bonn, and Sydney respectively on the plot on the left.

The plot on the right shows that solar irradiance has a similar shape based on local hours throughout the day in each city. The solar irradiance starts to increase from zero at around 5 or 6 am in each city, it peaks around noon and continues to decline before reaching zero again at 5 or 6 pm. On this day of the year, Sydney received the highest solar irradiance, followed by Chitwan, New York, and Bonn.

In this post, I demonstrated the methods to deal with time zones while working with datetime objects including dataframe in Python. I used the example of working with solar irradiance data for four cities across the world. These methodologies could be very handy while working with time series data, where time zones matter such as meteorological data. I have summarized the key techniques learnt from this post to deal with time zones in Python in the following numbered bullets:

It is possible to check the time zone of a datetime object using tzinfo module.

2. When the datetime object does not contain any information about time zones and daylight saving, it is called naive datetime object.

3. Using the timezone module of pytz package, it is possible to convert naive time to local time. For example,

time_in_nepal =  timezone("Asia/Kathmandu”).localize(datetime.now())

4. The new object is now time zone aware. It is possible to get the time in a different time zone using astimezone module of datetime object. For example,

german_timezone = timezone(“Europe/Berlin”)
time_in_germany = time_in_nepal.astimezone(german_timezone)

5. To work with time series data, it makes sense to convert the index of pandas dataframe to datetime index.

6. The naive dataframe index can be localized using tz_localize module in df and specifying the time zone. For example,

df_utc = df.tz_localize(tz = “UTC”)

7. The dataframe object can also be converted to different time zone using tz_convert module of df.

df_nepal = df_utc.tz_convert(tz = “Asia/Kathmandu”)

The data, code and output plots for this post are available in notebooks/Timezone_handling folder in this GitHub repository. Thank you for reading!