Skip to content

Commit 19cde1c

Browse files
Add schema.lineage property to view all lineages
- Add get_schema_lineages() function in lineage.py - Add schema.lineage property returning flat dict mapping 'schema.table.attribute' to its lineage origin - Add note about A - B without semantic check in spec - Document schema.lineage in API reference 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
1 parent a3dcabf commit 19cde1c

File tree

3 files changed

+47
-0
lines changed

3 files changed

+47
-0
lines changed

docs/src/design/semantic-matching-spec.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,19 @@ if schema.lineage_table_exists:
128128

129129
**Returns**: `bool` - `True` if `~lineage` table exists, `False` otherwise.
130130

131+
#### `schema.lineage`
132+
133+
Property returning all lineage entries for the schema.
134+
135+
```python
136+
schema.lineage
137+
# {'myschema.session.session_id': 'myschema.session.session_id',
138+
# 'myschema.trial.session_id': 'myschema.session.session_id',
139+
# 'myschema.trial.trial_num': 'myschema.trial.trial_num'}
140+
```
141+
142+
**Returns**: `dict` - Maps `'schema.table.attribute'` to its lineage origin
143+
131144
### Join Methods
132145

133146
#### `expr.join(other, semantic_check=True)`
@@ -174,6 +187,8 @@ Equivalent to `A.restrict(B, semantic_check=True)`.
174187

175188
Restriction with negation. Semantic checking applies.
176189

190+
To bypass semantic checking: `A.restrict(dj.Not(B), semantic_check=False)`
191+
177192
#### `A + B` (Union)
178193

179194
Union of expressions. Requires all namesake attributes to have matching lineage.

src/datajoint/lineage.py

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,26 @@ def get_table_lineages(connection, database, table_name):
109109
return {row[0]: row[1] for row in results}
110110

111111

112+
def get_schema_lineages(connection, database):
113+
"""
114+
Get all lineages for a schema from the ~lineage table.
115+
116+
:param connection: A DataJoint connection object
117+
:param database: The schema/database name
118+
:return: A dict mapping 'schema.table.attribute' to its lineage
119+
"""
120+
if not lineage_table_exists(connection, database):
121+
return {}
122+
123+
results = connection.query(
124+
"""
125+
SELECT table_name, attribute_name, lineage FROM `{database}`.`~lineage`
126+
""".format(database=database),
127+
).fetchall()
128+
129+
return {f"{database}.{table}.{attr}": lineage for table, attr, lineage in results}
130+
131+
112132
def insert_lineages(connection, database, entries):
113133
"""
114134
Insert multiple lineage entries in the ~lineage table as a single transaction.

src/datajoint/schemas.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -333,6 +333,18 @@ def lineage_table_exists(self):
333333
self._assert_exists()
334334
return lineage_table_exists(self.connection, self.database)
335335

336+
@property
337+
def lineage(self):
338+
"""
339+
Get all lineages for tables in this schema.
340+
341+
:return: A dict mapping 'schema.table.attribute' to its lineage
342+
"""
343+
from .lineage import get_schema_lineages
344+
345+
self._assert_exists()
346+
return get_schema_lineages(self.connection, self.database)
347+
336348
def rebuild_lineage(self):
337349
"""
338350
Rebuild the ~lineage table for all tables in this schema.

0 commit comments

Comments
 (0)