The “Ghost in the Machine”: Tracking Variable Names and Metadata in PyTorch

#Bypass PyTorch’s attribute guards: Use stack inspection and __setattr__ to ‘stamp’ YAML metadata directly onto your dynamic model instances.

In Python, an object doesn’t know its own name. The interpreter my_model = L2Regularization() creates an object at a memory address, and it my_model is merely a pointer. But when you are parsing a complex YAML configuration with multiple regularizers, knowing which "key" created which "instance" becomes a debugging necessity.

#Here is the deep dive into how to retrieve names using stack inspection and how to force metadata into the PyTorch ecosystem.

#1. The “Detective” Approach: Stack Inspection

If you want to find the name of a variable as it exists in your code, you have to look at the Execution Stack. Every time a function runs, Python creates a Frame containing two vital dictionaries:

• f_locals: Variables defined inside the current function scope.
• f_globals: Variables defined at the module (top) level.

The “Sophisticated” Inspector

To find an instance’s name, we must look at the Caller’s Frame (f_back). In the terminal (REPL), f_locals and f_globals are often identical because you are already at the top level.

import inspect
def get_var_name(obj):
    # Retrieve the frame of the code that called this function
    frame = inspect.currentframe().f_back
    
    # If called from a terminal/REPL, f_back might be None
    if frame is None:
        frame = inspect.currentframe()
    # In the terminal, locals and globals are the same. 
    # In a script, we must check both to be thorough.
    search_space = {**frame.f_globals, **frame.f_locals}
    
    try:
        for name, value in search_space.items():
            if value is obj:
                return name
    finally:
        del frame # Crucial to prevent memory leaks (Reference Cycles)
    return None

The Catch: This only works if the object has a unique variable name. If your objects are inside a list (like [Reg(), Reg()]), they have no "name" in the namespace, and this method returns None.

#2. The “Stamping” Approach: Metadata Injection

When stack inspection fails (common in YAML-driven loops), you must inject the name into the object. However, doing this with torch.nn.Module is tricky.

Why setattr fails in PyTorch

PyTorch overrides the standard __setattr__. It expects every new attribute to be a Parameter or a Buffer. If you try to assign a raw string like instance.yaml_key = "l2_target", PyTorch’s internal logic sometimes intercepts this and—if the timing isn't perfect—throws an AttributeError.

The Surgical Bypass: object.__setattr__

To force a metadata “stamp” onto a PyTorch instance without triggering its internal guards, we call the method from the root of all Python objects:

# Bypass the L2Regularization/PyTorch __setattr__ entirely
object.__setattr__(reg_instance, 'yaml_key', 'l2_regularization_target')

# OR directly as
reg_instance.__setattr__('yaml_key', 'l2_regularization_target')

This writes the key directly into the object’s__dict__, bypassing any "locked" logic or specialized PyTorch checks.

#3. The Retrieval Paradox

Even after a successful injection, you might encounter this error:

AttributeError: 'L2Regularization' object has no attribute 'yaml_key'

This is the ultimate PyTorch “Ghost.” Even though the data is in the object’s memory, PyTorch __getattr__ is programmed to look for tensors. If it doesn't see your attribute in its "authorized list", it claims it doesn't exist.

The Solution: Using****vars()

To read your “stamped” metadata, bypass the class getter and look at the raw dictionary:

# Don't do: reg.yaml_key (PyTorch might block this)
# Do this:
name = vars(reg).get('yaml_key')

# Or using __dict__
name = reg.__dict__.get('yaml_key')

vars(reg) or reg.__dict__ returns the underlying dictionary of the instance. Because we used __setattr__ to put the data there, vars() or __dict__is the only 100% reliable way to get it back out.

#Summary for the Sophisticated Developer

1. For Variable Names: Use inspect to merge f_locals and f_globals from the caller's frame.
2. For YAML Keys: Use object.__setattr__ to "force-stamp" the instance during the loop.
3. For Retrieval: Use vars(obj) to peek behind the PyTorch curtain and avoid AttributeError.

The “Ghost in the Machine”: Tracking Variable Names and Metadata in PyTorch was originally published in The Owl on Medium, where people are continuing the conversation by highlighting and responding to this story.

Originally published on Medium.