|
| 1 | +# Existing Pipelines |
| 2 | + |
| 3 | +This section describes how to work with database schemas without access to the original |
| 4 | +code that generated the schema. These situations often arise when the database is |
| 5 | +created by another user who has not shared the generating code yet or when the database |
| 6 | +schema is created from a programming language other than Python. |
| 7 | + |
| 8 | +## Loading Classes |
| 9 | + |
| 10 | +Typically, a DataJoint schema is created as a dedicated Python module. This module |
| 11 | +defines a schema object that is used to link classes declared in the module to tables |
| 12 | +in the database schema. With the module installed, you can simply import it to interact |
| 13 | +with its tables: |
| 14 | + |
| 15 | +``` python |
| 16 | +import datajoint as dj |
| 17 | +from element_calcium_imaging import scan # (1) |
| 18 | +``` |
| 19 | + |
| 20 | +1. This and other [DataJoint Elements](https://datajoint.com/docs/elements/) are |
| 21 | +installable via `pip` or downloadable via their respective GitHub repositories. |
| 22 | + |
| 23 | +To visualize an unfamiliar schema, see commands for generating [diagrams](../../getting-started/#diagram). |
| 24 | + |
| 25 | +## Spawning Missing Classes |
| 26 | + |
| 27 | +Now, imagine we do not have access to the |
| 28 | +[Python definition of Scan](https://github.com/datajoint/element-calcium-imaging/blob/main/element_calcium_imaging/scan.py), |
| 29 | +or we're unsure if the version on our server matches the definition available. We can |
| 30 | +use the `dj.list_schemas` function to list the available database schemas. |
| 31 | + |
| 32 | +``` python |
| 33 | +import datajoint as dj |
| 34 | +dj.conn() # (1) |
| 35 | +dj.list_schemas() # (2) |
| 36 | +dj.Schema('schema_name').list_tables() # (3) |
| 37 | +``` |
| 38 | + |
| 39 | +1. Establish a connection to the server. |
| 40 | +2. List the available schemas on the server. |
| 41 | +3. List the tables for a given schema from the previous step. These will appear in their |
| 42 | +raw database form, with underscores instead of camelcase and special characters for Part |
| 43 | +tables. |
| 44 | + |
| 45 | +Just as with a new schema, we can create a schema object to connect to the chosen |
| 46 | +database schema. If the schema already exists, `dj.Schema` is initialized as usual. |
| 47 | + |
| 48 | +If a diagram will shows a mixture of class names and database table names, the |
| 49 | +`spawn_missing_classes` method will spawn classes into the local namespace for any |
| 50 | +tables missing their classes. This will allow us to interact with all tables as if |
| 51 | +they were declared in the current namespace. |
| 52 | + |
| 53 | +``` python |
| 54 | +schema.spawn_missing_classes() |
| 55 | +``` |
| 56 | + |
| 57 | +## Virtual Modules |
| 58 | + |
| 59 | +While `spawn_missing_classes` creates the new classes in the local namespace, it is |
| 60 | +often more convenient to import a schema with its Python module, equivalent to the |
| 61 | +Python command. We can mimmick this import without having access to the schema using |
| 62 | +the `VirtualModule` class object: |
| 63 | + |
| 64 | +```python |
| 65 | +import datajoint as dj |
| 66 | +subject = dj.create_virtual_module(module_name='subject', schema_name='db_subject') |
| 67 | +``` |
| 68 | + |
| 69 | +Now, `subject` behaves as an imported module complete with the schema object and all the |
| 70 | +table classes. |
| 71 | + |
| 72 | +The class object `VirtualModule` of the `dj.Schema` class provides access to virtual |
| 73 | +modules. It creates a python module with the given name from the name of a schema on |
| 74 | +the server, automatically adds classes to it corresponding to the tables in the |
| 75 | +schema. |
| 76 | + |
| 77 | +The function can take several parameters: |
| 78 | + |
| 79 | +- `module_name`: displayed module name. |
| 80 | + |
| 81 | +- `schema_name`: name of the database in MySQL. |
| 82 | + |
| 83 | + `create_schema`: if `True`, create the schema on the database server if it does not |
| 84 | + already exist; if `False` (default), raise an error when the schema is not found. |
| 85 | + |
| 86 | +- `create_tables`: if `True`, `module.schema` can be used as the decorator for declaring |
| 87 | + new classes; if `False`, such use will raise an error stating that the module is |
| 88 | + intend only to work with existing tables. |
| 89 | + |
| 90 | +The function returns the Python module containing classes from the schema object with |
| 91 | +all the table classes already declared inside it. |
| 92 | + |
| 93 | +`create_schema=False` may be useful if we want to make sure that the schema already |
| 94 | +exists. If none exists, `create_schema=True` will create an empty schema. |
| 95 | + |
| 96 | +``` python |
| 97 | +dj.VirtualModule('what', 'nonexistent') |
| 98 | +``` |
| 99 | + |
| 100 | +Returns |
| 101 | + |
| 102 | +``` python |
| 103 | +DataJointError: Database named `nonexistent` was not defined. Set argument create_schema=True to create it. |
| 104 | +``` |
| 105 | + |
| 106 | +`create_tables=False` prevents the use of the schema object of the virtual module for |
| 107 | +creating new tables in the existing schema. This is a precautionary measure since |
| 108 | +virtual modules are often used for completed schemas. `create_tables=True` will new |
| 109 | +tables to the existing schema. A more common approach in this scenario would be to |
| 110 | +create a new schema object and to use the `spawn_missing_classes` function to make the |
| 111 | +classes available. |
| 112 | + |
| 113 | +However, you if do decide to create new tables in an existing tables using the virtual |
| 114 | +module, you may do so by using the schema object from the module as the decorator for |
| 115 | +declaring new tables: |
| 116 | + |
| 117 | +``` python |
| 118 | +uni = dj.VirtualModule('university.py', 'dimitri_university', create_tables=True) |
| 119 | +``` |
| 120 | + |
| 121 | +``` python |
| 122 | +@uni.schema |
| 123 | +class Example(dj.Manual): |
| 124 | + definition = """ |
| 125 | + -> uni.Student |
| 126 | + --- |
| 127 | + example : varchar(255) |
| 128 | + """ |
| 129 | +``` |
| 130 | + |
| 131 | +``` python |
| 132 | +dj.Diagram(uni) |
| 133 | +``` |
0 commit comments