Environment variable (e.g., BATCH_JOIN_TIMEOUT) is read as string, causing TypeError in comparisons

[gurgenyegor]

When running Snuba in Kubernetes, environment variables (such as BATCH_JOIN_TIMEOUT) are always provided as strings. However, Snuba’s code appears to compare them as if they were integers, leading to an error:

TypeError: '>' not supported between instances of 'str' and 'int'

Steps to reproduce

1. Deploy Snuba (getsentry/snuba) in a Kubernetes environment.
2. Set the environment variable BATCH_JOIN_TIMEOUT=120 (or any numeric value) in the Deployment spec.
3. Observe Snuba logs on startup or during runtime, where a TypeError is raised when comparing the string to an integer.

[gurgenyegor]

My pods is restarting with this error:

2025-02-25 10:03:28,258 Processor terminated
Traceback (most recent call last):
  File "/usr/local/bin/snuba", line 33, in <module>
    sys.exit(load_entry_point('snuba', 'console_scripts', 'snuba')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/snuba/snuba/cli/consumer.py", line 279, in consumer
    consumer.run()
  File "/usr/local/lib/python3.11/site-packages/arroyo/processing/processor.py", line 323, in run
    self._run_once()
  File "/usr/local/lib/python3.11/site-packages/arroyo/processing/processor.py", line 397, in _run_once
    self.__processing_strategy.poll()
  File "/usr/local/lib/python3.11/site-packages/arroyo/processing/strategies/healthcheck.py", line 32, in poll
    self.__next_step.poll()
  File "/usr/local/lib/python3.11/site-packages/arroyo/processing/strategies/filter.py", line 69, in poll
    self.__next_step.poll()
  File "/usr/local/lib/python3.11/site-packages/arroyo/processing/strategies/guard.py", line 101, in poll
    self.__inner_strategy.poll()
  File "/usr/local/lib/python3.11/site-packages/arroyo/processing/strategies/run_task.py", line 55, in poll
    self.__next_step.poll()
  File "/usr/local/lib/python3.11/site-packages/arroyo/processing/strategies/guard.py", line 37, in poll
    self.__next_step.poll()
  File "/usr/local/lib/python3.11/site-packages/arroyo/processing/strategies/reduce.py", line 101, in poll
    self.__buffer_step.poll()
  File "/usr/local/lib/python3.11/site-packages/arroyo/processing/strategies/buffer.py", line 169, in poll
    self.__next_step.poll()
  File "/usr/local/lib/python3.11/site-packages/arroyo/processing/strategies/run_task_in_threads.py", line 107, in poll
    result = future.result()
             ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/snuba/snuba/consumers/strategy_factory.py", line 135, in flush_batch
    message.payload.close()
  File "/usr/src/snuba/snuba/consumers/consumer.py", line 300, in close
    self.__insert_batch_writer.close()
  File "/usr/src/snuba/snuba/consumers/consumer.py", line 136, in close
    self.__writer.write(
  File "/usr/src/snuba/snuba/clickhouse/http.py", line 361, in write
    batch.join(timeout=batch_join_timeout)
  File "/usr/src/snuba/snuba/clickhouse/http.py", line 243, in join
    response = self._result.result(timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 458, in result
    raise TimeoutError()
TimeoutError

and I tried add BATCH_JOIN_TIMEOUT but its not work :(

[volokluev]

Thanks for the report! This will be fixed in the next version