Skip to content

Memory leak in python binding #4713

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
korzck opened this issue May 2, 2025 · 0 comments
Open

Memory leak in python binding #4713

korzck opened this issue May 2, 2025 · 0 comments
Labels
Bug Bug in learning semantics, critical by default

Comments

@korzck
Copy link

korzck commented May 2, 2025

Describe the bug

Currently building service that operates with large number of vw models. I noticed memory increase for each model load iteration and checked for vw heap size.

Memory before:
420471734272
Memory after few iterations:
420633214976

This small diff gives constant increase in memory

How to reproduce

import psutil
import vowpalwabbit
NEW_MODEL_CONFIG = f"""
    -i model.bin --cb_explore_adf --quiet --epsilon 0.1 -q UA -q UC \
    --l2 0.002 \
    --learning_rate 0.5 \
    --power_t 0.5
    """

def my_func():
    model = vowpalwabbit.Workspace(NEW_MODEL_CONFIG)
    
    ex = model.parse(
        """
        shared |User user_id=123 user_name=John Doe [email protected] user_age=30 user_gender=male
        0:-1.0:1 |Action variant=variant1 
        |Action variant=variant2 
        |Action variant=variant3 
        """,
        vowpalwabbit.LabelType.CONTEXTUAL_BANDIT,
    )
    model.learn(ex)
    model.finish()
    del ex
    del model

if __name__ == '__main__':
    print(psutil.Process().memory_full_info().vms)
    for i in range(100):
        my_func()
    print(psutil.Process().memory_full_info().vms)
    

which prints smth like this:

420471734272
420633214976

Version

9.10.0

OS

macOS

Language

Python

Additional context

No response

@korzck korzck added the Bug Bug in learning semantics, critical by default label May 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Bug in learning semantics, critical by default
Projects
None yet
Development

No branches or pull requests

1 participant