[Partial Feature] Table name as field instead of Pragma

[gzigurella]

Feature Request

Is your feature request related to a problem?

Trying to dynamically add tables to my ORM based CMS

Describe the solution you'd like

Add a check on Model fields at norm/postgres.nim(134, 96) to see if the object contains the field with the key table, in which case we use it to create the table name

Describe alternatives you've considered

I've considered manipulating nim AST, forking the project for my own purpose, yet I still think this could be an useful feature allowing to expand the usage of norm for web-development along with frameworks like Jester or Prologue

Teachability, Documentation, Adoption, Migration Strategy

Norm's Users may use this feature to reduce duplicated code, keeping cleaner their work and allowing them to test more easily without having to re-build the software.

Conversion from JSON to SQL Table could be easily achieved as well wrapping Model inside a JsonModel

How can norm benefit from this feature in the future? Norm could become the Nim ORM standard by de facto implementing a core feature to build Search Engines and similar on top of this feature see Elasticsearch Json Processor for example

[moigagoo]

@gzigurella Hi! So you propose there's a special reserved field name "table" or "tableName" for each Model, similar to how every Model has "id"?

What is the advantage of this versus the pragma?

[moigagoo]

I mean if you're generating the code for your models anyway, you can generate the pragma too.

[gzigurella]

@moigagoo your assumption on my proposal is correct.

The main difference is the time of use, pragma is expected to be used at compile-time giving instructions to compiler without cluttering the source code.

I suggest to use a special field to make use of the ORM functionalities at runtime , this way new models can be defined at runtime with input-based fields.

In my example I've created a variant model but with the pragma I am able to define it just once, while I may need different tables unknown at compile time but known at runtime through serialization and deserialization.

This way we can fully have a bidirectional ORM (Database-to-Object and Object-to-Database), the first one static and known at compile-time while the latter could be dynamic and allows a better integration with dynamic languages such as JavaScript and Python.

I think checking for this special field could be definetely easier than messing with Nim AST to introduce dynamic pragmas.

tl;dr:
Benefit comes from the use at runtime versus compile-time, which leads to better support of serialization and deserialization of tables from dynamic languages and weak typed languages.

[moigagoo]

@gzigurella I think I'm misunderstanding something. Sorry, I'll have to ask more questions :-)

AFAIK if you're generating types, it must be done at compile time anyway. You can't generate a type at runtime. And if you're generating at compile time, it's templates or macros anyway, which means generating a type definition with pragma is about as easy as without one.

Could you please share some code you have? It definitely seems we're talking about different things.

[gzigurella]

@moigagoo it's okay don't worry, maybe I'm using the wrong terminology since I'm not an english native speaker 😄

I'm not currently on the workstation i use to write in Nim, but I'll try to make a reasonable example in the meanwhile.

I agree with you, it's not possible to generate types at runtime, what is possible is to create a proxy like Java reflection does, therefore we cannot define a static type but we can indeed define a type that manipulates the JSON payload it receives, or another kind of serialized object, and mirrors operations onto the database.

I refer to JSON as example because is the one I linked in the first post of this Issue. As you can see from the JsonModel

Using the pragma is possible to create just one of these proxy objects mirrored on the database, since the pragma get evaluated at compile-time and it cannot be changed after.

Therefore if I want to have n proxy objects as tables defined on the database I have to know them at compile-time, which is not always possible.

That's why I suggested to add the table name as a special field in the Model object. There are some drawbacks like excessive overhead to manipulate the proxy, yet it allows to a functional use of dynamic types that are not provided in Nim lang, since it's statically typed.

The most "dynamic" approach to types that Nim offers are Object Variants as far as I know, implementing my suggestion allows to operate easily in cases like this where the object is not known at compile-time but is known at runtime, either because the system we operate with is not in our control or is made up with dynamic and weak types (like JavaScript for example)

For future reference I paste here the JsonModel code, this is a Variant Model I thought to persist and interact with the database through ORM and a RESTful architecture.

import norm/[model, pragmas]
import std/[json, tables, sequtils]
import uuids
 
template evalJSON(arg_type: string, fieldName: string, table: Table[string, string]) =
    case arg_type:
        of "string":
            table[fieldName] = "string"
        of "integer":
            table[fieldName] = "int"
        of "bool":
            table[fieldName] = "bool"
        of "number":
            table[fieldName] = "float"
 
template eval*(value: string, arg_type: string) {.dirty.} =
    var result = nil
    case arg_type:
        of "string":
            result = value
        of "integer":
            result = parseBiggestInt(value)
        of "bool":
            result = parseBool(value)
        of "number":
            result = parseFloat(value)
 
type
    JsonModel* = ref object of Model
        external_reference_uuid* : string #! External Reference UUID useful to crete URI to point at the Object
        table*: string  #! SQL Table Name
        fieldsType*: string #! Json HashTable
        fieldsValue*: string #! Json HashTable
        keys* : string #! Json Array
 
proc newJsonModel*(PK: string, VIEW: string, TABLE_TYPES: string, TABLE_VALUES: string, FIELD_NAMES: string) : VariantModel =
    return JsonModel(external_reference_uuid: PK, table: VIEW, fieldsType: TABLE_TYPES, fieldsValue: TABLE_VALUES, keys: FIELD_NAMES)
 
proc newJsonModel*(tableName: string) : JsonModel =
    var 
        typeTable = initTable[string, string]()
        valueTable = initTable[string, string]()
    var
        jsonTypes : JsonNode = %typeTable
        jsonValues : JsonNode = %valueTable
    return newJsonModel($genUUID(), tableName, $jsonTypes, $jsonValues, "[]")
 
proc newJsonModel*(JSON : JsonNode) : JsonModel =
    if JSON.kind == JObject:
        let uuid = $genUUID() #* Generate UUID for the new Database Table
        var keysArg : seq[string]
        for k in JSON.keys: #* Store all JSON keys as named_fields for the Database Table
            keysArg.add(k)
        var fieldsType = initTable[string, string]()
        var fieldsValue = initTable[string, string]()
        for i in JSON["cols"]:
            let name = i["fieldname"].getStr(uuid&"_field_"&"MISSING_NAME")
            let typeJS = i["type"].getStr("string")
            evalJSON(typeJS, name, fieldsType)
            fieldsValue[name] = i["value"].getStr()
        let jsonTypeMap = %fieldsType
        let jsonValueMap = %fieldsValue
        return newJsonModel(uuid, JSON["table"].getStr(uuid&"_table"), $jsonTypeMap, $jsonValueMap, $(%keysArg))
    else:
        return nil
 
template `[]`*(self: JsonModel, key : string) =
    #! Use in case of non-nullable values
    let typeMap : JsonNode = parseJson(self.fieldsType)
    let valueMap : JsonNode = parseJson(self.fieldsValue)
    eval(valueMap[key], typeMap[key])
    result {.inject.} = e
 
template `{}`*(self: JsonModel, key : string) =
    #! Use in case of nullable values
    let fields = self.keys
    if fields.filter(proc(e:string) : bool = e == key).len == 1:
        self[key]
    else:
        result {.inject.} = nil
 
template `!`* (self: JsonModel, key: string) : string =
    #! Use for Debug purpose
    let typeMap : JsonNode = parseJson(self.fieldsType)
    echo typeMap[key]