Skip to content

Conversation

@yash-dayama
Copy link

Fix Memory Leaks in Console and HTTP Transports

Description

This PR fixes memory leak issues in both the Console and HTTP transports that occur when logging a large number of messages in quick succession. The issues were reported in #2548 and #2465, affecting Winston 3.x.x versions.

Problems

  1. Console Transport:

    • Using setImmediate to emit the 'logged' event created many pending operations
    • Under high load, these operations weren't being cleaned up properly
  2. HTTP Transport:

    • Callback handling could create unnecessary async operations
    • Batch mode could accumulate requests without proper cleanup
    • Event listeners weren't being properly managed

Solutions

  1. Console Transport:

    • Removed unnecessary setImmediate wrapper since console writes are already async
    • Events are now emitted directly, reducing memory overhead
  2. HTTP Transport:

    • Improved callback handling to prevent async operation buildup
    • Enhanced batch mode cleanup
    • Added proper event listener management
    • Optimized request handling

Changes

  • Removed setImmediate wrapper in Console transport's log method
  • Improved HTTP transport's batch handling and cleanup
  • Added comprehensive stress tests for both transports
  • Added event listener cleanup verification

Testing

Added new stress tests that:

  1. Verify memory usage remains stable when logging many messages
  2. Ensure event listeners are properly cleaned up
  3. Test batch mode behavior in HTTP transport

To run the tests:

# Run with garbage collection enabled for memory tests
node --expose-gc node_modules/mocha/bin/mocha test/unit/winston/transports/console*.test.js
node --expose-gc node_modules/mocha/bin/mocha test/unit/winston/transports/http*.test.js

Related Issues

Backwards Compatibility

These changes maintain full backwards compatibility as they only optimize internal operations without changing the transports' APIs or behaviors.

Yash Dayama and others added 5 commits July 29, 2025 14:42
Copy link
Contributor

@DABH DABH left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few questions and comments here

log(info, callback) {
// Handle callback immediately since HTTP requests are already async
if (callback) {
return callback();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would we return here if there's a call back? We would then exit the function before anything is logged?

_doBatch(options, callback, auth, path) {
// Handle callback immediately since batching is async
if (callback) {
return callback();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same q here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please revert changes to this file

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please delete this file from this PR as it's part of your other PR #2564

// First message stored, start the timeout
this.batchTimeoutID = setTimeout(() => {
this.batchTimeoutID = -1;
this._doBatchRequest(null, auth, path);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is null passed rather than callback?

// max batch count reached, send immediately
clearTimeout(this.batchTimeoutID);
this.batchTimeoutID = -1;
this._doBatchRequest(null, auth, path);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same q here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Possible Memory leak? [Bug]: Memory leak happens when running stress test to send logs with winston and its http transport

2 participants