Skip to content

Atlas does not recognise schema files when an __init__.py file/explicit namespace is present #27

Open
@DevJake

Description

@DevJake

I currently have the following file structure in my project (stripped out irrelevant files):

.
├── infrastructure
│   ├── db-schemas
│   │   ├── README.md
│   │   ├── atlas.hcl
│   │   ├── docker-compose.yml
│   │   ├── migrations
│   │   │   ├── 20250409153652.sql
│   │   │   ├── 20250409153727.sql
│   │   │   ├── 20250409173449.sql
│   │   │   ├── 20250409173742.sql
│   │   │   ├── 20250411151352.sql
│   │   │   ├── 20250415151031.sql
│   │   │   ├── 20250429185315.sql
│   │   │   └── atlas.sum
│   │   ├── poetry.lock
│   │   └── pyproject.toml
├── services
│   └── Conversation
│       ├── poetry.lock
│       ├── pyproject.toml
│       └── src
│           ├── app.py
│           ├── db.py
│           ├── schema
│           │   ├── __init__.py
│           │   └── schema.py

A schema for the database is defined inside schema.py (excerpt):

class Message(DefaultSchemaModel):
    __tablename__ = 'messages'
    transcript_id: Mapped[int] = mapped_column(
        ForeignKey('transcripts.id'),
        nullable=False,
        comment='The ID of the conversation this message belongs to',
    )

    role: Mapped[Role] = mapped_column(
        Enum(Role),
        nullable=False,
        comment='The role of the sender of the message',
    )

    content: Mapped[str] = mapped_column(
        Text,
        nullable=False,
        comment='The content of the message',
    )

    transcript: Mapped['Transcript'] = relationship(
        back_populates='messages',
        lazy='selectin',
    )


class Transcript(DefaultSchemaModel):
    __tablename__ = 'transcripts'
    assistant_id: Mapped[int] = mapped_column(
        ForeignKey('assistants.id'),
        nullable=False,
        comment='The ID of the assistant that this transcript belongs to',
    )

    messages: Mapped[list['Message']] = relationship(
        back_populates='transcript',
        lazy='selectin',
        cascade='all, delete-orphan',
        order_by='desc(Message.created_at)',
    )

    assistant: Mapped['Assistant'] = relationship(
        back_populates='transcripts',
        lazy='selectin',
    )

And the schema/__init__.py file:

from .schema import *

I then point the atlas.hcl file to this schema directory, like so:

data "external_schema" "sqlalchemy" {
  program = [
    "poetry",
    "run",
    "atlas-provider-sqlalchemy",
    "--path", "../../services/Conversation/src/schema",
    "--dialect", "postgresql",
  ]
}


env ephemeral {
  src = data.external_schema.sqlalchemy.url
  url = "docker://postgres/17/app?search_path=public"
  dev = "docker://postgres/17/app?search_path=public"

  exclude = ["atlas_schema_revisions"]

  migration {
    dir = "file://migrations"
  }

  format {
    migrate {
      diff = "{{ sql . }}"
    }
  }
}

When I execute atlas migrate diff --env ephemeral, I get the following error:

Error: data.external_schema.sqlalchemy: running program /home/jake/.local/bin/poetry: ImportError: attempted relative import with no known parent package in ../../services/Conversation/src/schema/__init__.py
To skip on failed import, run: atlas-provider-sqlalchemy --skip-errors

When deleting the __init__.py file as aforementioned, the command runs just fine and the new migration is generated. This is also fixed by removing the relative import inside the __init__.py file. However, removing that import is not ideal, as it's used to structure the import system libraries use.

As a workaround, I can of course use --skip-errors and the migration will be generated, but this is a blanket error ignore and will mask any other issues present.

Is there a more reasonable workaround to this, or is a code change required?

Many thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions