ValueError while running alembic upgrade head with kubectl exec -it my_pod -- alembic upgrade head

[itsgigahex]

Describe the bug
ValueError while running alembic upgrade head with kubectl exec -it my_pod -- alembic upgrade head

Expected behavior
database migration should happen

To Reproduce

I m running dockerized streamlit app with alembic and postgressql , and the database is running in a different ecs environment,
after deploying the changes and try to do the migration getting the error,

import os
from logging.config import fileConfig

from sqlalchemy import engine_from_config, create_engine
from sqlalchemy import pool

from alembic import context

# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
from models import Base

config = context.config

# Interpret the config file for Python logging.
# This line sets up loggers basically.
if config.config_file_name is not None:
    fileConfig(config.config_file_name)

# add your model's MetaData object here
# for 'autogenerate' support
# from myapp import mymodel
target_metadata = Base.metadata


# target_metadata = None


# other values from the config, defined by the needs of env.py,
# can be acquired:
# my_important_option = config.get_main_option("my_important_option")
# ... etc.
def get_url():
    return (
        f'postgresql+psycopg2://{os.getenv("DB_USER")}:{os.getenv("DB_PASS")}@{os.getenv("DB_HOST")}:5432/'
        f'{os.getenv("DB_NAME")}'
    )


config.set_main_option("sqlalchemy.url", get_url())


def run_migrations_offline() -> None:
    """Run migrations in 'offline' mode.

    This configures the context with just a URL
    and not an Engine, though an Engine is acceptable
    here as well.  By skipping the Engine creation
    we don't even need a DBAPI to be available.

    Calls to context.execute() here emit the given string to the
    script output.

    """
    print("running offline migrations")

    url = config.get_main_option("sqlalchemy.url")

    context.configure(
        url=url,
        target_metadata=target_metadata,
        literal_binds=True,
        dialect_opts={"paramstyle": "named"},
    )

    with context.begin_transaction():
        context.run_migrations()


def run_migrations_online() -> None:
    """Run migrations in 'online' mode.

    In this scenario we need to create an Engine
    and associate a connection with the context.

    """
    print("running online migrations")
    connectable = engine_from_config(
        config.get_section(config.config_ini_section, {}),
        prefix="sqlalchemy.",
        poolclass=pool.NullPool,
    )

    with connectable.connect() as connection:
        context.configure(connection=connection, target_metadata=target_metadata)

        with context.begin_transaction():
            context.run_migrations()


if context.is_offline_mode():
    run_migrations_offline()
else:
    run_migrations_online()

Error

INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
Traceback (most recent call last):
  File "/usr/local/bin/alembic", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/alembic/config.py", line 641, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/usr/local/lib/python3.10/site-packages/alembic/config.py", line 631, in main
    self.run_cmd(cfg, options)
  File "/usr/local/lib/python3.10/site-packages/alembic/config.py", line 608, in run_cmd
    fn(
  File "/usr/local/lib/python3.10/site-packages/alembic/command.py", line 403, in upgrade
    script.run_env()
  File "/usr/local/lib/python3.10/site-packages/alembic/script/base.py", line 583, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/usr/local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/bumblebee/alembic/env.py", line 95, in <module>
    run_migrations_online()
  File "/bumblebee/alembic/env.py", line 89, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/local/lib/python3.10/site-packages/alembic/runtime/environment.py", line 948, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/lib/python3.10/site-packages/alembic/runtime/migration.py", line 615, in run_migrations
    for step in self._migrations_fn(heads, self):
  File "/usr/local/lib/python3.10/site-packages/alembic/command.py", line 392, in upgrade
    return script._upgrade_revs(revision, rev)
  File "/usr/local/lib/python3.10/site-packages/alembic/script/base.py", line 459, in _upgrade_revs
    for script in reversed(list(revs))
  File "/usr/local/lib/python3.10/site-packages/alembic/script/revision.py", line 813, in iterate_revisions
    revisions, heads = fn(
  File "/usr/local/lib/python3.10/site-packages/alembic/script/revision.py", line 1438, in _collect_upgrade_revisions
    for rev in self._parse_upgrade_target(
  File "/usr/local/lib/python3.10/site-packages/alembic/script/revision.py", line 1229, in _parse_upgrade_target
    return self.get_revisions(target)
  File "/usr/local/lib/python3.10/site-packages/alembic/script/revision.py", line 545, in get_revisions
    resolved_id, branch_label = self._resolve_revision_number(id_)
  File "/usr/local/lib/python3.10/site-packages/alembic/script/revision.py", line 767, in _resolve_revision_number
    self._revision_map
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 1140, in __get__
    obj.__dict__[self.__name__] = result = self.fget(obj)
  File "/usr/local/lib/python3.10/site-packages/alembic/script/revision.py", line 210, in _revision_map
    for revision in self._generator():
  File "/usr/local/lib/python3.10/site-packages/alembic/script/base.py", line 155, in _load_revisions
    script = Script._from_filename(self, dir_name, filename)
  File "/usr/local/lib/python3.10/site-packages/alembic/script/base.py", line 1042, in _from_filename
    module = util.load_python_file(dir_, filename)
  File "/usr/local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 879, in exec_module
  File "<frozen importlib._bootstrap_external>", line 1017, in get_code
  File "<frozen importlib._bootstrap_external>", line 947, in source_to_code
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
ValueError: source code string cannot contain null bytes

Versions.

• OS: Ubuntu
• Python: 3.10.12
• Alembic: 1.13.1
• SQLAlchemy: =2.0.28

Have a nice day! any help would be appreciated

[zzzeek]

check out the container you are deploying, you are missing .py files in your versions/ directory, or they are corrupted, or .pyc files are corrupted, or some permissions issue, something like that.

[itsgigahex]

I have tried checking for any null values, then added init.py.
Migration files are generated in MacOs and I m deploying it in the Ubuntu, is that could be an issue ?

[zzzeek]

maybe? see if you can ensure those source files are normalized to have newlines, not carriage returns.

[itsgigahex]

i have checked not carriage returns.
is anything to do with the utf8 ?

[itsgigahex]

i m creating this files in Mac Os and docker is in linux, hope that wont create any issues

[itsgigahex]

I finally got it working, these files are generated in macOS to create backup file for files in ._.filename and that was causing the issue, manually removed these files and build the image again and then everything works perfectly.

while checking the the migration files in the revision folder its taking all the files in that folder i guess, if we can do something about it that will be great