Path to professional use

[shanemmattner]

Over the last few years I've been spinning a ton of PCB's for different designs at work. Many times I've wished the circuits were defined in Python code to avoid mistakes like accidentally connecting nets or accidentally having multiple 3.3v power rails (+3.3v, +3v3, etc). Lately I've been making more complex boards that are a huge pain to check that every pin of a 300+ pin chip is connected properly. It would be nice to have those large circuits in code so it's easy to double-check connections. Another big motivator to use Python based circuits is being able to feed a circuit into an LLM for analysis and feedback. With the advent of tools like Quilter it seems like a natural fit to generate entire circuit boards from Python defined code. Other companies are trying to make circuits with code (Jitx) but SKiDL just seems so much more simple and approachable using Python and easy syntax.

So I've been thinking about what would need to change or be added to SKiDL so I could practically use it in my professional day to day PCB designs. It seems like there's just a few features that would be needed so SKiDL could be really useful to me.

• Schematic generation: I don't think fully human like schematic generation is necessary, see my comment in the other thread.
• Ability to update existing KiCAD projects. Currently, I believe that SKiDL always generates a complete new PCB or schematic when a script is run. I would need the ability for SKiDL to know what the current design is in the KiCAD project, then only update parts that are added or modified.
• Convert KiCAD projects to SKiDL. This feature would likely be part of the update logic described above.

Let me know what you think @devbisme . I have time, interest, and tokens to work on these problems if you could provide some guidance and feedback along the way.

[bhusang]

@shanemmattner Not wanting to diminish the usefulness of SKiDL (which I used and appreciated before I joined JITX), but your point about schematics not needing to be human-like may miss the mark for lots of folks. We have repeatedly seen that a significant fraction of the ecosystem wants or needs access to a usable schematic. When you take PCBs into production using a code-based approach (like SKiDL or JITX), there are still many touch points that need traditional artifacts (i.e. schematics, etc).

[shanemmattner]

@bhusang thank you for your comment and your point is certainly valid: having a readable schematic is necessary for a professional design and it's hard to know when you will need to analyze a circuit from a schematic viewpoint when moving through the production process.

I think long term SKiDL should have auto placement/routing of parts and beautification of schematics to human level. Dave worked on schematic generation and routing for ~2 years and got some pretty good results with simpler circuits. I'm not sure this algorithm would scale for more complex circuits with lots of connections.

But I think if we can do the 2nd feature I mention of updating the already existing project between SKiDL script executions then we could allow a user to move, connect, and rotate components as they want and keep the schematic layout persistent. This workflow wouldn't be as automatic as a complete circuit generation, but it's a short term fix that I think could work in practice and allow SKiDL to be used professionally sooner than waiting to completely solve the schematic generation problem. It looks like JitX has a Sr Software engineer dedicated to just schematic generation so it's got to be a hard problem.

[devbisme]

• Schematic generation: I don't think fully human like schematic generation is necessary, see my comment in the other thread.

I think the current PCB design ecosystem does need human-readable schematics. It's simply too entrenched in the community. The only way to move off schematics is by showing another technique that makes it faster/easier to generate designs with fewer errors, and even then it might require a generation. This happened with the FPGA/ASIC community when they switched over to HDLs, but they didn't have a long history of using schematics and many of those designers already had coding backgrounds.

• Ability to update existing KiCAD projects. Currently, I believe that SKiDL always generates a complete new PCB or schematic when a script is run. I would need the ability for SKiDL to know what the current design is in the KiCAD project, then only update parts that are added or modified.

You're correct about this, and it's been a problem I've worried about for years. Recently, an issue (#238) referenced a related problem that has made me think there is a possible solution, but it will require some manual input into the SKiDL code from the designer. So that's something I'll be working on.

• Convert KiCAD projects to SKiDL. This feature would likely be part of the update logic described above.

There is the existing netlist_to_skidl utility to convert a KiCad netlist file into a Python script that describes the circuit in SKiDL syntax. I'll have to check that it still works with the latest versions of KiCad. The problem is that it outputs a linear list of components followed by a list of net connections without arranging anything into a coherent whole or re-creating the hierarchy (if there was one).

I have time, interest, and tokens to work on these problems...

I have interest in addressing some of these issues. I think your efforts to apply LLMs to SKiDL have the most potential and we should continue that. I can look what it will take to keep SKiDL output consistent with the KiCad PCB (issue #238), and hammer netlist_to_skidl back into shape so that it at least works.

Schematics, however, are a time suckhole. I think SKiDL should output SVG instead of KiCad schematics. SVG is more universal in case SKiDL gets used in a non-KiCad environment. The other problem with KiCad schematics is that a user will want to edit them and then have those changes back-annotated into the SKiDL code. That's very hard if the SKiDL code is using things like iterative loops, subcircuits, groups, etc. and not just a linear list of statements. It's like editing machine code and then trying to automatically go back and update the C code to generate it.

It looks like JitX has a Sr Software engineer dedicated to just schematic generation so it's got to be a hard problem.

Geez, that's a sweet gig! I worked 21 months on schematics and didn't even get a fucking tee-shirt, although I did design one:

[devbisme]

This feature could be useful but I think realistically for professional use we'd need the schematics to be able to be parsed and preserve all the hierarchy information.

I took a look into a KiCad 8 .netlist file and it records the hierarchical information in the sheetpath field of each component. So netlist_to_skidl could be modified to rebuild the schematic hierarchy in the generated SKiDL code. Is there other information in the .kicad_sch files that you're using? (People put crazy amounts stuff into KiCad schematics with embedded images, spatially-arranged text fields, etc. I can see the appeal if the schematic is the main source of design documentation, but that's got to be a nightmare to parse and recreate as SKiDL code.)

[shanemmattner]

This feature could be useful but I think realistically for professional use we'd need the schematics to be able to be parsed and preserve all the hierarchy information.

I took a look into a KiCad 8 .netlist file and it records the hierarchical information in the sheetpath field of each component. So netlist_to_skidl could be modified to rebuild the schematic hierarchy in the generated SKiDL code. Is there other information in the .kicad_sch files that you're using? (People put crazy amounts stuff into KiCad schematics with embedded images, spatially-arranged text fields, etc. I can see the appeal if the schematic is the main source of design documentation, but that's got to be a nightmare to parse and recreate as SKiDL code.)

Parsing .netlist files into SKiDL files is a good idea. I'll likely reuse some of the logic I'm making to parse kicad schematics.

I got the kicad schematic parsing mostly working with kiutils. Here's an example text file describing a esp32 board with usb, esp32, 5v to 3v regulator, and version control resistor divider that was generated by analyze kicad schematics starting with a top hierarchical sheet:

esp32_example_hierarchy_circuit.txt

I'm now working on logic to stitch that information in the text file together into a SKiDL based project. So far it's going well. There's definately edge cases my logic doesn't cover yet but the core parsing seems to work. And kiutils allows me to add/remove and move components as well so it may do all the annoying kicad work with s-expressions.

[shanemmattner]

I see now what you were trying to say @devbisme; I mistakenly thought the netlist file was an output from PCBNEW. The netlist file seems to have all information I want. Maybe kiutils will be helpful in the future but for now I'll use the netlist file as you suggest.

I tried importing the netlist to skidl through netlist_to_skidl -i my_design.net -o my_design.py -w and got the output below. Is it possible to have the output be a hierarchical representation of the circuit instead of flat parts and connections? I think ideally the generated SKiDL project would match the KiCAD project structure.

# -*- coding: utf-8 -*-

from skidl import *


def _Users_shanemattner_Desktop_skidl_kicad_schematic_parser_example_kicad_project_example_kicad_project_kicad_sch():

    #===============================================================================
    # Component templates.
    #===============================================================================

    Connector_Generic_Conn_02x03_Odd_Even_Connector_IDC_IDC_Header_2x03_P2_54mm_Vertical = Part('Connector_Generic', 'Conn_02x03_Odd_Even', dest=TEMPLATE, footprint='Connector_IDC:IDC-Header_2x03_P2.54mm_Vertical')
    setattr(Connector_Generic_Conn_02x03_Odd_Even_Connector_IDC_IDC_Header_2x03_P2_54mm_Vertical, 'Footprint', 'Connector_IDC:IDC-Header_2x03_P2.54mm_Vertical')
    setattr(Connector_Generic_Conn_02x03_Odd_Even_Connector_IDC_IDC_Header_2x03_P2_54mm_Vertical, 'Datasheet', '')
    setattr(Connector_Generic_Conn_02x03_Odd_Even_Connector_IDC_IDC_Header_2x03_P2_54mm_Vertical, 'Description', 'Generic connector, double row, 02x03, odd/even pin numbering scheme (row 1 odd numbers, row 2 even numbers), script generated (kicad-library-utils/schlib/autogen/connector/)')

    Connector_USB_C_Plug_USB2_0_Connector_USB_USB_C_Receptacle_GCT_USB4105_xx_A_16P_TopMnt_Horizontal = Part('Connector', 'USB_C_Plug_USB2.0', dest=TEMPLATE, footprint='Connector_USB:USB_C_Receptacle_GCT_USB4105-xx-A_16P_TopMnt_Horizontal')
    setattr(Connector_USB_C_Plug_USB2_0_Connector_USB_USB_C_Receptacle_GCT_USB4105_xx_A_16P_TopMnt_Horizontal, 'Footprint', 'Connector_USB:USB_C_Receptacle_GCT_USB4105-xx-A_16P_TopMnt_Horizontal')
    setattr(Connector_USB_C_Plug_USB2_0_Connector_USB_USB_C_Receptacle_GCT_USB4105_xx_A_16P_TopMnt_Horizontal, 'Datasheet', 'https://www.usb.org/sites/default/files/documents/usb_type-c.zip')
    setattr(Connector_USB_C_Plug_USB2_0_Connector_USB_USB_C_Receptacle_GCT_USB4105_xx_A_16P_TopMnt_Horizontal, 'Description', 'USB 2.0-only Type-C Plug connector')

    Device_C = Part('Device', 'C', dest=TEMPLATE)
    setattr(Device_C, 'Footprint', '')
    setattr(Device_C, 'Datasheet', '')
    setattr(Device_C, 'Description', 'Unpolarized capacitor')

    Device_C_Capacitor_SMD_C_0603_1608Metric = Part('Device', 'C', dest=TEMPLATE, footprint='Capacitor_SMD:C_0603_1608Metric')
    setattr(Device_C_Capacitor_SMD_C_0603_1608Metric, 'Footprint', 'Capacitor_SMD:C_0603_1608Metric')
    setattr(Device_C_Capacitor_SMD_C_0603_1608Metric, 'Datasheet', '')
    setattr(Device_C_Capacitor_SMD_C_0603_1608Metric, 'Description', 'Unpolarized capacitor')

    Device_R_Resistor_SMD_R_0603_1608Metric = Part('Device', 'R', dest=TEMPLATE, footprint='Resistor_SMD:R_0603_1608Metric')
    setattr(Device_R_Resistor_SMD_R_0603_1608Metric, 'Footprint', 'Resistor_SMD:R_0603_1608Metric')
    setattr(Device_R_Resistor_SMD_R_0603_1608Metric, 'Datasheet', '')
    setattr(Device_R_Resistor_SMD_R_0603_1608Metric, 'Description', 'Resistor')

    RF_Module_ESP32_S3_MINI_1_RF_Module_ESP32_S2_MINI_1 = Part('RF_Module', 'ESP32-S3-MINI-1', dest=TEMPLATE, footprint='RF_Module:ESP32-S2-MINI-1')
    setattr(RF_Module_ESP32_S3_MINI_1_RF_Module_ESP32_S2_MINI_1, 'Footprint', 'RF_Module:ESP32-S2-MINI-1')
    setattr(RF_Module_ESP32_S3_MINI_1_RF_Module_ESP32_S2_MINI_1, 'Datasheet', 'https://www.espressif.com/sites/default/files/documentation/esp32-s3-mini-1_mini-1u_datasheet_en.pdf')
    setattr(RF_Module_ESP32_S3_MINI_1_RF_Module_ESP32_S2_MINI_1, 'Description', 'RF Module, ESP32-S3 SoC, Wi-Fi 802.11b/g/n, Bluetooth, BLE, 32-bit, 3.3V, SMD, onboard antenna')

    Regulator_Linear_NCP1117_3_3_SOT223_Package_TO_SOT_SMD_SOT_223_3_TabPin2 = Part('Regulator_Linear', 'NCP1117-3.3_SOT223', dest=TEMPLATE, footprint='Package_TO_SOT_SMD:SOT-223-3_TabPin2')
    setattr(Regulator_Linear_NCP1117_3_3_SOT223_Package_TO_SOT_SMD_SOT_223_3_TabPin2, 'Footprint', 'Package_TO_SOT_SMD:SOT-223-3_TabPin2')
    setattr(Regulator_Linear_NCP1117_3_3_SOT223_Package_TO_SOT_SMD_SOT_223_3_TabPin2, 'Datasheet', 'http://www.onsemi.com/pub_link/Collateral/NCP1117-D.PDF')
    setattr(Regulator_Linear_NCP1117_3_3_SOT223_Package_TO_SOT_SMD_SOT_223_3_TabPin2, 'Description', '1A Low drop-out regulator, Fixed Output 3.3V, SOT-223')


    #===============================================================================
    # Component instantiations.
    #===============================================================================

    C1 = Device_C_Capacitor_SMD_C_0603_1608Metric(ref='C1', value='C')

    C10 = Device_C(ref='C10', value='100nF')

    C11 = Device_C(ref='C11', value='C')

    C2 = Device_C_Capacitor_SMD_C_0603_1608Metric(ref='C2', value='10uF')

    C3 = Device_C_Capacitor_SMD_C_0603_1608Metric(ref='C3', value='10uF')

    C4 = Device_C(ref='C4', value='10uF')

    C9 = Device_C(ref='C9', value='C')

    J1 = Connector_Generic_Conn_02x03_Odd_Even_Connector_IDC_IDC_Header_2x03_P2_54mm_Vertical(ref='J1', value='Conn_02x03_Odd_Even')

    P1 = Connector_USB_C_Plug_USB2_0_Connector_USB_USB_C_Receptacle_GCT_USB4105_xx_A_16P_TopMnt_Horizontal(ref='P1', value='USB_C_Plug_USB2.0')

    R1 = Device_R_Resistor_SMD_R_0603_1608Metric(ref='R1', value='5.1K')

    R10 = Device_R_Resistor_SMD_R_0603_1608Metric(ref='R10', value='1k')

    R11 = Device_R_Resistor_SMD_R_0603_1608Metric(ref='R11', value='R')

    R2 = Device_R_Resistor_SMD_R_0603_1608Metric(ref='R2', value='R')

    R7 = Device_R_Resistor_SMD_R_0603_1608Metric(ref='R7', value='R')

    R8 = Device_R_Resistor_SMD_R_0603_1608Metric(ref='R8', value='R')

    R9 = Device_R_Resistor_SMD_R_0603_1608Metric(ref='R9', value='2k')

    U1 = Regulator_Linear_NCP1117_3_3_SOT223_Package_TO_SOT_SMD_SOT_223_3_TabPin2(ref='U1', value='NCP1117-3.3_SOT223')

    U3 = RF_Module_ESP32_S3_MINI_1_RF_Module_ESP32_S2_MINI_1(ref='U3', value='ESP32-S3-MINI-1')


    #===============================================================================
    # Net interconnections between instantiated components.
    #===============================================================================

    Net('+3V3').connect(C1['1'], C2['1'], J1['2'], R8['1'], R9['1'], U1['2'], U3['3'])

    Net('+5V').connect(C3['1'], C4['1'], P1['A4'], P1['A9'], P1['B4'], P1['B9'], R2['1'], U1['3'])

    Net('/3v3_monitor').connect(C11['1'], R11['1'], R8['2'], U3['6'])

    Net('/5v_monitor').connect(C9['1'], R2['2'], R7['1'], U3['7'])

    Net('/D+').connect(P1['A6'], U3['24'])

    Net('/D-').connect(P1['A7'], U3['23'])

    Net('/esp32s3mini1/EN').connect(J1['1'], U3['45'])

    Net('/esp32s3mini1/HW_VER').connect(C10['1'], R10['1'], R9['2'], U3['5'])

    Net('/esp32s3mini1/IO0').connect(J1['6'], U3['4'])

    Net('/esp32s3mini1/RX').connect(J1['5'], U3['40'])

    Net('/esp32s3mini1/TX').connect(J1['3'], U3['39'])

    Net('GND').connect(C1['2'], C10['2'], C11['2'], C2['2'], C3['2'], C4['2'], C9['2'], J1['4'], P1['A1'], P1['A12'], P1['B1'], P1['B12'], P1['S1'], R1['2'], R10['2'], R11['2'], R7['2'], U1['1'], U3['1'], U3['2'], U3['42'], U3['43'], U3['46'], U3['47'], U3['48'], U3['49'], U3['50'], U3['51'], U3['52'], U3['53'], U3['54'], U3['55'], U3['56'], U3['57'], U3['58'], U3['59'], U3['60'], U3['61'], U3['62'], U3['63'], U3['64'], U3['65'])

    Net('Net-(P1-CC)').connect(P1['A5'], R1['1'])

    Net('unconnected-(P1-VCONN-PadB5)').connect(P1['B5'])

    Net('unconnected-(U3-IO10-Pad14)').connect(U3['14'])

    Net('unconnected-(U3-IO11-Pad15)').connect(U3['15'])

    Net('unconnected-(U3-IO12-Pad16)').connect(U3['16'])

    Net('unconnected-(U3-IO13-Pad17)').connect(U3['17'])

    Net('unconnected-(U3-IO14-Pad18)').connect(U3['18'])

    Net('unconnected-(U3-IO15-Pad19)').connect(U3['19'])

    Net('unconnected-(U3-IO16-Pad20)').connect(U3['20'])

    Net('unconnected-(U3-IO17-Pad21)').connect(U3['21'])

    Net('unconnected-(U3-IO18-Pad22)').connect(U3['22'])

    Net('unconnected-(U3-IO21-Pad25)').connect(U3['25'])

    Net('unconnected-(U3-IO26-Pad26)').connect(U3['26'])

    Net('unconnected-(U3-IO33-Pad28)').connect(U3['28'])

    Net('unconnected-(U3-IO34-Pad29)').connect(U3['29'])

    Net('unconnected-(U3-IO35-Pad31)').connect(U3['31'])

    Net('unconnected-(U3-IO36-Pad32)').connect(U3['32'])

    Net('unconnected-(U3-IO37-Pad33)').connect(U3['33'])

    Net('unconnected-(U3-IO38-Pad34)').connect(U3['34'])

    Net('unconnected-(U3-IO39-Pad35)').connect(U3['35'])

    Net('unconnected-(U3-IO4-Pad8)').connect(U3['8'])

    Net('unconnected-(U3-IO40-Pad36)').connect(U3['36'])

    Net('unconnected-(U3-IO41-Pad37)').connect(U3['37'])

    Net('unconnected-(U3-IO42-Pad38)').connect(U3['38'])

    Net('unconnected-(U3-IO45-Pad41)').connect(U3['41'])

    Net('unconnected-(U3-IO46-Pad44)').connect(U3['44'])

    Net('unconnected-(U3-IO47-Pad27)').connect(U3['27'])

    Net('unconnected-(U3-IO48-Pad30)').connect(U3['30'])

    Net('unconnected-(U3-IO5-Pad9)').connect(U3['9'])

    Net('unconnected-(U3-IO6-Pad10)').connect(U3['10'])

    Net('unconnected-(U3-IO7-Pad11)').connect(U3['11'])

    Net('unconnected-(U3-IO8-Pad12)').connect(U3['12'])

    Net('unconnected-(U3-IO9-Pad13)').connect(U3['13'])


#===============================================================================
# Instantiate the circuit and generate the netlist.
#===============================================================================

if __name__ == "__main__":
    _Users_shanemattner_Desktop_skidl_kicad_schematic_parser_example_kicad_project_example_kicad_project_kicad_sch()
    generate_netlist()

[devbisme]

Maybe kiutils will be helpful in the future but for now I'll use the netlist file as you suggest.

The thing I liked about using the KiCad schematics and kiutils is that you got information about the positions of symbols. I can see using that to get training data to help create a better algorithm for generating schematics. But that's not on my immediate to-do list.

Another thing kiutils has is its sexpr function that can parse a netlist file. I'll probably extract that and use it in netlist_to_skidl because it runs so much faster than kinparse (which uses pyparsing).

Is it possible to have the output be a hierarchical representation of the circuit instead of flat parts and connections?

Yes, the sheetpath field of each component in the netlist file stores the hierarchy information. I would initially try to embed the hierarchy using nested SKiDL Group blocks rather than making separate subcircuits. Once that's working, we could try for a more complicated subcircuit implementation.

[shanemmattner]

I took a shot at refactoring the netlist import logic in this PR #243

This new logic should mirror the structure of the original kicad files and make it easy to integrate with the new LLM logic, as well as set the foundation for round-trip import-export from KiCAD to SKiDL I describe above in this thread. I couldn't get the SKiDL netlist to match the original netlist as solving this bug will be required.

[tapegoji]

Hi,
I want to add something here. I think 100% with this statement form Dave.

"I think the current PCB design ecosystem does need human-readable schematics. It's simply too entrenched in the community. The only way to move off schematics is by showing another technique that makes it faster/easier to generate designs with fewer errors, and even then it might require a generation. This happened with the FPGA/ASIC community when they switched over to HDLs,"

As a power electronics engineer, I think I can skip the schematic generation. I usually have a netlist for simulation and then I convert that into a pcb.

I think we can go from SKiDL to PCB generation directly. The challenge is when the PCB is generated by SKiDL the parts are not properly placed. Hiereplace does a good job but it was breaking on the new KiCAD version, due to different timestmps. I don't know if that's fixed.

I was able to use some GA to improve the placement of the components. Essentially, a GA enabled kinet2pcb module. However, I got busy and left it there. Flux (the company is doing it)

What do you guys think of working on that area?

[devbisme]

Hierplace is its own separate project independent of SKiDL. If you want to improve it, go ahead! That would benefit not only SKiDL, but KiCad as well.

(I'll have to check hierplace to see if it's compatible with the current version of KiCad with its unstable Python API.)

[tapegoji]

Last time I checked, it was not working with SKiDL generated netlists. because the format of timestamp changed in KiCad 7 I think. The timestamp for components and sheets are generated with SKiDL. You had a recent commit about timestamp and that might have fixed it. Have not tested it yet.