Are there alternatives to Malm for common issues?
I have been using MALM for some time across several projects, but I keep encountering recurring problems, especially regarding implementation stability and performance bottlenecks in larger applications. Therefore, I am wondering if there are well-established alternatives to MALM that are more robust or efficient in handling these issues. It would be important to find tools or frameworks that offer similar functionality but potentially scale better or allow for simpler error management. I look forward to your experiences and recommendations.
I have been using MALM for some time across several projects, but I keep encountering recurring problems, especially regarding implementation stability and performance bottlenecks in larger applications. Therefore, I am wondering if there are well-established alternatives to MALM that are more robust or efficient in handling these issues. It would be important to find tools or frameworks that offer similar functionality but potentially scale better or allow for simpler error management. I look forward to your experiences and recommendations.
For an in-depth analysis of your issue: In asynchronous microservices, logging is one of the most critical components, especially when complete traceability needs to be ensured.
Malm sometimes suffers from backpressure when logs are generated quickly but processed slowly.
Some points you should check:
- Are there any blockages in buffer pools?
- What is the flush frequency?
- Which sinks are being used? Local or remote?
For example, Serilog offers better handling of backpressure with its Async Sink. This means logs can be buffered and written asynchronously to different systems without blocking the main process.
Separating critical error logs and standard logs into different pipelines can also help balance the load more effectively.
It is also important to check instrumentation tools for specific metrics during your peak load. Once you identify bottlenecks, you can make targeted adjustments or switch to an alternative.
What exactly are you currently using as a sink? File, database, ELK stack?
Malm sometimes suffers from backpressure when logs are generated quickly but processed slowly.
Some points you should check:
- Are there any blockages in buffer pools?
- What is the flush frequency?
- Which sinks are being used? Local or remote?
For example, Serilog offers better handling of backpressure with its Async Sink. This means logs can be buffered and written asynchronously to different systems without blocking the main process.
Separating critical error logs and standard logs into different pipelines can also help balance the load more effectively.
It is also important to check instrumentation tools for specific metrics during your peak load. Once you identify bottlenecks, you can make targeted adjustments or switch to an alternative.
What exactly are you currently using as a sink? File, database, ELK stack?
L
Leandrop5729 Jul 2018 13:18Currently, data is mainly written to local files, which are then picked up by a log shipper and fed into the central ELK system.
So it could be that MALM does not complete the file output quickly enough, yes. Thanks – that matches the observations.
Async Sink in Serilog sounds very interesting. Would that be the first recommendation in such a setup to minimize data loss under high load?
So it could be that MALM does not complete the file output quickly enough, yes. Thanks – that matches the observations.
Async Sink in Serilog sounds very interesting. Would that be the first recommendation in such a setup to minimize data loss under high load?
Leandrop57 schrieb:
Currently, logging is mainly done to local filesThis is a typical bottleneck, especially with high logging rates.
Therefore:
1. Use Serilog with an Async File Sink, or even better: direct syslog or network sinks if your ELK stack supports them.
2. Categorize your logs by priority: write critical errors synchronously, and everything else asynchronously.
3. Check the rolling interval of the log files. Very small files cause more frequent write operations.
4. If possible, decouple logging from the main process, for example, by using log queues.
These tips improve not only performance but also stability under peak loads.
poted schrieb:
Also important: Check instrumentation tools for specific metrics during your peak load.Exactly, before completely replacing MALM, you should analyze the hotspots using profiler tools and telemetry. It is often a misconfiguration or incorrectly sized hardware.
Guyju schrieb:
Divide your logs into priority classesI see that as a critical point. "A framework that tries to cover everything at once will never be stable."
Leandrop57, I would advise you to completely rethink your logging strategy before seriously considering alternatives.
Hey everyone :-)
Let’s bring a bit of color to the topic: I really like MALM, but yes, it has its quirks. I’ve tried both Serilog and NLog – and I have to say, Serilog is actually easier to handle for microservices, especially because of its API and extensive plugins.
But don’t forget, logging can also be fun 🎉. Once you’ve set it up, working with structured logs can make troubleshooting much more enjoyable!
Maybe try building a prototype with Serilog? It really feels very fresh!
Have you tried it yet?
Let’s bring a bit of color to the topic: I really like MALM, but yes, it has its quirks. I’ve tried both Serilog and NLog – and I have to say, Serilog is actually easier to handle for microservices, especially because of its API and extensive plugins.
But don’t forget, logging can also be fun 🎉. Once you’ve set it up, working with structured logs can make troubleshooting much more enjoyable!
Maybe try building a prototype with Serilog? It really feels very fresh!
Have you tried it yet?
Similar topics