Python Logging - Saving Logs to a File & Sending Logs to an API
Get your logs to where they are needed to debug your code
Including logging into your Python app is essential for knowing what your program does and quick debugging. It allows you to solve an error as quickly and easily as possible. Your program can be logging out useful information but how can it notify you when something goes wrong? We cannot read the logs off the terminal in a crashed app!
This article shows you two ways of saving your logs using handlers. The first is the simplest; just writing all logs to a file using the FileHandler. The second uses a HttpHandler to send your logs to an HTTP endpoint (like an API).
Let’s first understand the concept of using handlers when logging. As you might know, you can log a message by creating a logger and then calling one of the logging methods on that logger like below:
When we call the debug, info of error methods in the example above, our logger has to handle those logging calls. By default, it just prints the messages (with some meta-data as specified with the format in the basicConfig) to the console.
In the parts below we add more handlers to the logger that do other things with the logs. Per handler, we can specify the level, fields, and formats as we’ll see below.
In this article we’ll make it our goal to add three handlers to our logger that each has their own format:
- stream handler
For printing to the console. We want to print all logs (debugand up) - file handler
saves logs in a file. We want to save all logs exceptdebuglogs - HTTP handler
sends logs over HTTP (to an API for example). We want to send onlyerrorandcriticallogs
All of these loggers will be configured separately; they will have their own format and level.
1: Setting up our logger
First, we’ll create our logger, nice and simple:
Notice that we don’t do anything with basicConfig anymore. We set the default level with the setLevel method, next we’re re going to specify the formatting and level for each handler separately.
2. Adding a stream handler
We’ll configure our stream handler to send the log to our console, printing it out:
Notice that we first create a Formatter specifically for stream handler. Then we’ll define the actual StreamHandler, specifying that we want to output to sys.stdout (console) and then we set the formatter to the handler. Then we add the handler to the logging object.
The result:
We’ve successfully printed the log to our console! Check out how to create colored logging to your console with the article below:
3. Adding a file handler
The steps are exactly the same as with the stream handler. The differences are that we specify another format and a different level.
Results:
Opening the test.log file shows us that we haven’t just written a file, we can also clearly see that the log has a different format from the streamHandler. Also notice that we do not save the debug logs to the file. Exactly like we wanted.
Once you’ve used the filehandler for a while your logging file gets quite large. Check out this article to solve that problem.
4. Adding an HTTP Handler
The steps are not much different from the previous one:
These short few lines of code will send our log over HTTP to a URL.
IMPORTANT: This handler has one major downside, however: it blocks the thread until it has finished the request. In the future, I’ll focus on implementing a custom logging handler for a non-blocking HTTP handler. Follow me and stay tuned!
In this article, we’ve gone through handlers for streams, files, and HTTP. There are some others that we haven’t handled like the socket handler, and the SMTP handler. Because this article is already pretty lengthy I haven’t covered these in this article but maybe will in future ones so make sure to follow me.
I hope everything was as clear as I hope it to be but if this is not the case please let me know what I can do to clarify further. In the meantime, check out my other articles on all kinds of programming-related topics like these:
Happy coding!
— Mike
P.S: like what I’m doing? Follow me!
Get your logs to where they are needed to debug your code
Including logging into your Python app is essential for knowing what your program does and quick debugging. It allows you to solve an error as quickly and easily as possible. Your program can be logging out useful information but how can it notify you when something goes wrong? We cannot read the logs off the terminal in a crashed app!
This article shows you two ways of saving your logs using handlers. The first is the simplest; just writing all logs to a file using the FileHandler. The second uses a HttpHandler to send your logs to an HTTP endpoint (like an API).
Let’s first understand the concept of using handlers when logging. As you might know, you can log a message by creating a logger and then calling one of the logging methods on that logger like below:
When we call the debug, info of error methods in the example above, our logger has to handle those logging calls. By default, it just prints the messages (with some meta-data as specified with the format in the basicConfig) to the console.
In the parts below we add more handlers to the logger that do other things with the logs. Per handler, we can specify the level, fields, and formats as we’ll see below.
In this article we’ll make it our goal to add three handlers to our logger that each has their own format:
- stream handler
For printing to the console. We want to print all logs (debugand up) - file handler
saves logs in a file. We want to save all logs exceptdebuglogs - HTTP handler
sends logs over HTTP (to an API for example). We want to send onlyerrorandcriticallogs
All of these loggers will be configured separately; they will have their own format and level.
1: Setting up our logger
First, we’ll create our logger, nice and simple:
Notice that we don’t do anything with basicConfig anymore. We set the default level with the setLevel method, next we’re re going to specify the formatting and level for each handler separately.
2. Adding a stream handler
We’ll configure our stream handler to send the log to our console, printing it out:
Notice that we first create a Formatter specifically for stream handler. Then we’ll define the actual StreamHandler, specifying that we want to output to sys.stdout (console) and then we set the formatter to the handler. Then we add the handler to the logging object.
The result:
We’ve successfully printed the log to our console! Check out how to create colored logging to your console with the article below:
3. Adding a file handler
The steps are exactly the same as with the stream handler. The differences are that we specify another format and a different level.
Results:
Opening the test.log file shows us that we haven’t just written a file, we can also clearly see that the log has a different format from the streamHandler. Also notice that we do not save the debug logs to the file. Exactly like we wanted.
Once you’ve used the filehandler for a while your logging file gets quite large. Check out this article to solve that problem.
4. Adding an HTTP Handler
The steps are not much different from the previous one:
These short few lines of code will send our log over HTTP to a URL.
IMPORTANT: This handler has one major downside, however: it blocks the thread until it has finished the request. In the future, I’ll focus on implementing a custom logging handler for a non-blocking HTTP handler. Follow me and stay tuned!
In this article, we’ve gone through handlers for streams, files, and HTTP. There are some others that we haven’t handled like the socket handler, and the SMTP handler. Because this article is already pretty lengthy I haven’t covered these in this article but maybe will in future ones so make sure to follow me.
I hope everything was as clear as I hope it to be but if this is not the case please let me know what I can do to clarify further. In the meantime, check out my other articles on all kinds of programming-related topics like these:
Happy coding!
— Mike
P.S: like what I’m doing? Follow me!
Denial of responsibility! Techno Blender is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials, please contact us by email – [email protected]. The content will be deleted within 24 hours.