# cschem implementation - 2. Schematics data

## Goals

{imp2:0} This chapter should try to define the "first-class" objects cschem handles. The list of these objects should be as small as possible and as orthogonal as possible. All extra features and mechanisms, like slotting, hierarchy, name mangling, etc. should be implemented on top of these features (usually by dealing with attributes). This is in accordance with {pol:1}, {pol:2}, {pol:4}, {pol:9}

{imp2:1} The model can be considered as a stack:

       +------------------------+
       |  output, e.g. netlist  |
       +--+------------------+--+
          |  abstract model  |
       +--+------------------+--+
       |    concrete   model    |
    +--+------------------------+--+
    |      drawing  primitives     |
    |      and data schematics     |
    +------------------------------+

{imp2:2} The main objective should be to keep the abstract model as simple as possible, since most of the complex logics and methods and mechanisms will operate on this model.

{imp2:3} The concrete model can be more redundant, offering alternatives; the most common example is that a network can be realized drawing wire-segments, or drawing bus-segments or implying connections via attributes.

{imp2:4} The input for the concrete model can be even more elaborate; most notably the same concrete model elements can be created graphically or from data, manually or programatically.

{imp2:5} The final output is usually also a larger model that can be derived from the abstract model.

## Abstract vs. concrete

{imp2:6} At the end, all that matters is the abstract model. Most plugins, export formats, checkers, etc. are interested in that. That's what really describes the actual circuit. We have concrete objects to provide an easier access for the user to manipulate the abstract model.

{imp2:7} Electronic example: in the abstract model the project needs a quad NAND gate (7400). One of the possible implementations in the concrete model:

- in the lib, there are two symbols, a generic NAND gate with 3 gates and a TTL power symbol
- the user places an instance of the power symbol and 4 instances of the NAND symbol
- the users sets the name attribute of all 5 symbols to the same, unique value, e.g. U15
- cschem now knows that the abstract component called U15 depends on these five concrete symbols
- a slotting mechanism plugin and/or potentially other plugins affect (calculate) attributes of the five symbols and their ports (details to be discussed by later chapters)
- now that the final values are known, cschem attempts to combine the five concrete symbols into an abstract component (details to be discussed by later chapters)
- the rest of the code, e.g. netlist generator plugins, will often deal with abstract model only

{imp2:8} A much simpler example is when a single symbol instance of the concrete model becomes a single component in the abstract model. However, the above steps are all the same, except the 4 or 5 symbols are reduced to 1.

{imp2:9} The UI should present both the concrete and the abstract model. The user should be able to edit the objects of the concrete model and see the effect on the abstract model.

{imp2:10} Letting the user directly edit the abstract model is probably not practical: there are multiple ways the concrete model can be changed to cause the desired change in the abstract model. It is like if we wanted to fix a bug in the executable of a program and expect the debugger or compiler to go and fix the source code accordingly.

{imp2:11} Thus the UI should present the abstract model as read-only. But just like debuggers can assist the user to navigate to the source line(s) that contributed to a specific instruction or runtime situation, cschem UIs should be able to navigate the user to the source(s) in the concrete model that caused a specific effect in the abstract model. How the source references are stored in memory is up to the implementation, but when the model is saved/exported (e.g. to disk), it should use UIDs as specified.

{imp2:12} A common example is (abstract) networks. The user, looking at a PCB or simulation, wants to find the network named "Address6". This information is clearly interpreted in the scope of the abstract model, where there is a network with name "Address6" and a list of ports it connects (and a list of attributes). A search can be made to find all graphical connections, wire-nets, bus-nets, attribute connections and data that contributed to the final list of ports (and attributes) of the network.

{imp2:13} Note the terminology: the same terms are not reused in both the concrete and the abstract models, e.g. a symbol in the concrete model can become (or contribute to) a component in abstract model.

## Core attributes

{imp2:14} A very few aspects of the abstract model are implemented through attributes. These attributes are used by the core implementation and are relied upon by all plugins. These attributes are called the core attributes.

{imp2:15} The number of core attributes shall be as small as possible as per {imp2:16}, {imp2:17}, {imp2:18} and {imp2:19}.

{imp2:20} Currently the core attributes are restricted to deal with:

- identification of objects in the abstract and concrete models
- realizing the simplest forms of attribute based network connections (not excluding other, more complex forms calculated by plugins; the result of those calculations will be translated to connections specified in this simple form; scope of later chapters)

## Port implicit network vs. chan connection

{imp2:21} A port (or a channel of a bus-port) can be non-graphically connected to a network using the connect attribute. This always forms a connection (even at the cost of creating the given network).

{imp2:22} Another way is connecting the port graphically, using wire-nets in the concrete model.

{imp2:23} The two methods are orthogonal and can be used in parallel. If the same port is connected to two differently named networks, the usual network collision mechanism is used (will be discussed in a later chapter). If the port is connected to a named network via attribute and an unnamed wire-net, the wire-net automatically joins the named network (via the port).

{imp2:24} The third orthogonal idea is channels of a bus. A channel is really a network, which most often "stays within the scope of the bus", that is, won't have a connect attribute that would make it a globally known network. If a port is connected to a bus, the port's chan attribute determines which channel of the bus is connected.

{imp2:25} But this connection is again orthogonal to the above two. Which means a port can be connected to a network in multiple different ways:

- using the connect attribute of the port
- using the connect attribute of the parent component
- graphically, using a wire-net, assuming the wire-net is already associated with a network
- graphically, using a bus-net, assuming the bus-net channel is already connected to a network (via graphical connection to another port or wire-net or via the channel's connect attribute)

## Avoid attribute overloading

{imp2:26} A symbol has a name field for binding it to a component. This seems like redundant to refdes (also, gschem uses refdes for this purpose). The reason for this decision is that refdes is also the identifier printed on a PCB. In a complex hierarchic setup it makes more sense to use a shorter refdes generated from a longer name; the refdes can even be sequential.

{imp2:27} If cschem tried to use refdes for both, it would be sort of overloading the attribute with two different purposes:

- logical identification of the component in the hierarchy or on the schematics page
- physical identification of the component e.g. on a PCB

{imp2:28} The alternative is to provide separate attributes for the two purposes: name for the logical identification, refdes for the physical. Granting that:

- in the simplest, flat cases the refdes will just match the name
- refdes can be calculated or derived from the name by a plugin

{imp2:29} The same considerations are followed when netname is derived from name in case of networks.

## Proper bus support

{imp2:30} Buses are not pure graphical objects; cschem does understand what channels a bus has and does checks when anything is connected to a bus.

{imp2:31} Buses and bus-ports are first-class objects, which means they can be used the same way as networks:

- a bus can be connected to anything that accepts connections
- a component can be connected to a whole bus in a single connection, using a bus-port
- a bus-port in a hierarchy means buses can cross schematics sheet boundaries as buses

## UIDs vs. names

{imp2:32} While working on the back annotation for pcb-rnd -> gschem, I realized that using refdes for element identification is not reliable. Looking at prior art it turned out others struggled with the same problem (e.g. in the renumber plugin). The problem is: it's easy to communicate pin swaps and non-refdes-attribute changes, because we can address those by refdes:pinnum or refdes:attribname. But changing refdes (which should be just a plain attribute) is hard because it's also part of the addressing.

{imp2:33} In a gschem -> pcb-rnd flow this means if only the refdes attribute changes on the schematics e.g. from R15 to R21, pcb-rnd still preceives this as if R15 was removed and R21 was added. Not an attrib change, not a rename, but a full subcircuit removal and a new subcircuit added. The other way around is not easy either: when an subcircuit is renamed in pcb-rnd and needs to be back annotated: we have no other data to identify the subcircuits than its refdes, and the rest of the back annotation data pack also may depend on the refdes (e.g. for pin swaps in the same element).

{imp2:34} To fix this, cschem will have UIDs right from the start. Wherever possible in program-to-program communication, we will try to reference our objects by UIDs instead of names. Of course at the end the names are presented to the user, especially on the user interface, but the internal communication should be UID based. This makes it hard to change the UID of an existing object (which we don't want to do anyway) and makes it easy to change any other attribute, including the name of the object. This is an option; flows can decide to use any other identification from any other attribute specified by the user and/or generated by plugins

{imp2:35} Cschem will use the UIDs generated by libminuid.

{imp2:36} An abstract object is identified by its project-unique name attribute. The name attribute is (derived from) an attribute maintained by the user. This name may not be the same as the visible name on the output; for example in case of components the visible name on a PCB layout would be refdes attribute, in case of networks the PCB-visible name is the netname attribute while the unique name by which all systems identify the component or network is the name attribute. Refdes and netname can be derived automatically from name by plugins - the simplest method is copy.

## data/programmed schematics sheets

{imp2:37} Opaque data sheets gives cschem an extra level of flexibility: later on in a hierarchic or multisheet design, parts of a bigger schematics can be specified in:

- plain text data
- CSV, TSV, xml, json, binary data
- an awk or perl or python script that simply prints abstract objects on its stdout
- vhdl/verilog

{imp2:38} Any of these will need a plugin that handles the given format. For plain text, CSV, TSV xml, json, binary data it's just a format parser that then passes on the data to the same code that would handle the extracted concrete objects of a graphical sheet.

{imp2:39} For any kind of script, including vhdl and verilog, the plugin would need to run an external process or depend on a lib to actually execute the source code (that is the data of the sheet) and convert the result into concrete objects which then would be handled the same way as above.

{imp2:40} Since the sheet is opaque for the GUI, cschem is not forced to reproduce hundreds of editors and compilers/interpreters. Since the actual compiler is a plugin, the user can easily add support for a new language/format.

{imp2:41} The difference between a data schematics sheet and an external converter program that would compile the input into a graphical schematics sheet is:

- the data does not need to be converted to native cschem graphical sheets
- the data is handled from within cschem, by plugins; this makes the whole process feel more integrated
- the plugin has an option to provide a graphical representation, which doesn't need to look like cschem's GUI

{imp2:42} The data/programmed schematics sheet is an alternative to external conversions. Both methods can be used in parallel, even within the same project, as long as the resulting abstract model is consistent.