arche/mccoffee

The McCOFFEE plugin

PYCOGEN's concepts in arche-lang: a Melody CCM model in and out, and code for the McCOFFEE component framework. archec ships it in lib/plugins/mccoffee/, so every project sees it without the Store (archec plugin list).

What it brings

| Declaration | Command | What it does | |---|---|---| | importer melody | archec import x.ccmproject -o folder | the whole model and the projects it refers to (see Melody below) | | exporter melody | archec export main.arche --format melody -o x.ccmproject | the model back as a .ccmproject PYCOGEN reads | | generator mccoffee-cpp | archec gen main.arche --target mccoffee-cpp | the C++ PYCOGEN writes for McCOFFEE (or for the STL): see [C++](#c-generator-mccoffee-cpp) | | generator mccoffee-rust | archec gen main.arche --target mccoffee-rust --option runtime=threads | a Cargo workspace: the model, a crate per component, a binary per process (see Rust below) |

Signal processing (algorithms, composites, metrics) is generated with the rest, in C++ and in Rust. tools/ builds McCOFFEE and runs a generated application in a Linux container; tests/ holds the models equivalent to McCOFFEE's own unit tests (neither ships in the npm package, the Store package or the VS Code extension).

Melody

melody/ reads and writes Melody CCM (docs/importing.md, "Melody CCM", has the table of what is what).

• import.ts: the entry; workspace.ts reads the files and resolves the

references (#id, ../../P/model/P.ccmproject#id, platform:/plugin/…); defs.ts writes the modules (types, interfaces, components, algorithms, composites, metrics); deploy.ts the deployments (systems, platforms, binaries, threads, allocators, places); xml.ts is the XML reader.

• export.ts: the generator model as XMI. Options: module (where what is

in no package goes), vendor (com.dummygroup.mde, what PYCOGEN reads, or com.thalesgroup.mde), sizes (attributes or elements).

• MM/: the library models (the predefined types, the interaction patterns).

Ids: an element keeps its Melody id in attr melody_id (and its name in attr melody_name when arche-lang's differs); the exporter writes it back, else an id made from the element's path. Import → export → import gives the same model. A reference to a project that is not found is a stub (Missing_<project>, attr melody_missing), written back as the reference it was.

Not carried over (a // melody: comment and a note): SCA (out of scope), CCM assemblies' inner deployments, user-defined interaction patterns, user allocator policies, annotations, source file descriptors, the deployments of the projects referred to (import each on its own). The exporter writes no behaviour (Melody holds no code) and one deployment (the active one).

Files

• plugin.arche: the declarations.
• melody/import.ts, melody/export.ts: the importer and the exporter.
• cpp/generate.ts: the C++ generator; core/: the component view it reads; tools/: McCOFFEE's container build; tests/: the acceptance models.
• rust/: the Rust generator (generate.ts; view.ts reads the model into pycogen's view; rt/ the runtimes).
• tests/rust/: McCOFFEE's ut_csintf_simple, ut_ff, ut_dataflow and ut_timers as models, with their test code.

C++ (generator mccoffee-cpp)

archec gen main.arche --target mccoffee-cpp                          # framework mccoffee (the default)
archec gen main.arche --target mccoffee-cpp --option framework=stl    # std:: only
lib/plugins/mccoffee/tools/mccoffee-run.sh generated/mccoffee-cpp     # build it against McCOFFEE, run it

The view (core/)

core/view.ts reads the generator model as PYCOGEN sees an application. It is language neutral: the Rust generator can read it too.

• Each component has its executor (<Comp>_impl, or prop implementation),

and its ports in PYCOGEN's twelve lists: facets (provides, responds, consumes, receives, serves, sinks) and receptacles (uses, requests, publishes, fires, calls, sources).

• A port's list comes from its pattern: provides/requires an interface

is client-server, pattern request-reply, pub-sub, service, dataflow, and no pattern is fire-and-forget. A port of a kind algorithm with no pattern is dataflow.

• Each process of the deployment is a node (NODE1; a process named

nodeN is NODEN; options mccoffee-cpp { node "NODE3" } says it). It has its instances, active objects (threads), allocators, connections (local or to another node) and the patterns it installs.

• A composite (an algorithm with parts) is its parts: one instance each,

named <composite>_<role>, its delegations resolved.

• core/names.ts has PYCOGEN's hash codes (hashcode, the name's CRC-32).

framework mccoffee

The files and names are PYCOGEN's. Its profile is the generator's options, with PYCOGEN's names (container_prefix, container_suffix, prefix_component_core_filename, components_implementations_ignored, enum_class, support_lifecycle_execution_interceptors, rtev_active, pubsub_central_repo, library_packages, build_add_*…). Set them on a binary or a process: options mccoffee-cpp { … }.

| Files | What they are | |---|---| | src/orbless/<M>/<M>_datatypes.h | a package's constants, aliases, enums, structs (a message is a struct) | | src/rteorb/<M>/<M>_datatypes_streamoperators. | their RteOrb serialization | | src/gen/<m>/interfaces/<I>.h, cnts/active_object/<I>_ao_interceptor., _params. | an interface, a call into an active object's queue, its arguments | | src/rteorb/<m>/interfaces/<I>_stub., _skel., _ext_factories. | the call across nodes | | src/gen/<P>/cmpts/<C>/business/<C>_Core., <C>_<port>_impl., <C>_Connections.h, <C>_portnames.h | the executor, its facets, its receptacles, its ports' names | | src/gen/<P>/cmpts/<C>/cnts/<C>_Container. | the envelope McCOFFEE deploys | | deployment/Instances.h, rt/activeobject_<T>.h, rt/memallocator_<A>.h, services.h | instances, threads, allocators, services | | deployment/processes/<node>/instances/<inst>_factory. | how an instance is built: activation, active object, connectors, interceptors, buffering | | deployment/processes/<node>/process.cpp | the node's main | | build/x86_linux/cmake/<node>/CMakeLists.txt | its build against the McCOFFEE libraries |

User code goes in PYCOGEN's regions, by PYCOGEN's names (// -- start of user code: Core::start, an operation's name in a facet). Put the files PYCOGEN wrote in the output folder before archec gen: their code comes into the new files. A region name that several files have (include declarations) takes only its own file's code. With no user code, a region holds the model's body, translated to C++.

A component with prop library = "timerscomp" is built elsewhere (a predefined McCOFFEE component, like the timers): only its instance factory is written. library_packages ["Timers"] says whose types and interfaces are not written either.

A thread with a period steps its parts' functions from a thread of the node.

framework stl

--option framework=stl writes the same component shape on std:: only, C++17 and CMake, with no McCOFFEE. Beside the code it writes pyco_rt/, a small runtime:

• an active object per thread of the deployment: a std::thread and its queue;
• a pub/sub broker by topic: is one level, * any number, {x} any value;
• a service registry by name;
• dataflow sources and sinks, with push and execution thresholds, history, a

most per run, and back-pressure: the source waits while the sink has too many samples not yet run.

A two-way client-server call runs on the callee's thread and is waited for; a oneway call is not. Each node is an executable that runs run_ms (2000 by default), then stops and exits 0 (1 when a component's failures is not 0). When a connection crosses two nodes, all nodes run in one executable, app. Build it with cmake -S generated/mccoffee-cpp -B build && cmake --build build.

tools/

• tools/Containerfile: Debian bookworm with gcc 12 and CMake.
• tools/mccoffee-build.sh: builds McCOFFEE once, in the container, into

a cache folder (MCCOFFEE_CACHE, ~/.cache/arche-mccoffee by default; MCCOFFEE_TREE for a tree built already). It builds the framework's libraries, the test helpers, the synchronizer and the timers component.

• tools/mccoffee-run.sh <generated dir> [--build-only]: builds each node

in the container and runs it. Several nodes run beside McCOFFEE's deploy master and synchronizer (NODE0). The run passes when every node exits 0. It needs no archec.

Acceptance

tests/ has a model for each of McCOFFEE's own tests, with the user code of PYCOGEN's files (seed/):

| Model | McCOFFEE test | framework mccoffee (container) | framework stl (this Mac) | |---|---|---|---| | ut_csintf_simple_local | ut_csintf_simple/local | passes | exits 0 | | ut_csintf_simple_distributed | ut_csintf_simple/distributed | passes (2 nodes) | exits 0 (one app) | | ut_deployment_1coreNode0 | ut_deployment/1coreNode0 | passes | exits 0 | | ut_ff_local | ut_ff/local | passes | exits 0 | | ut_timers | ut_timers | passes | exits 0 (no timer thread) | | ut_dataflow_distributed | ut_dataflow/distributed | passes (2 nodes) | exits 0 | | sp_chain | examples/mccoffee's chain | passes | exits 0 |

test/mccoffee-cpp.test.js checks them; MCCOFFEE_CONTAINER=1 builds and runs them in the container.

Not done yet

• PYCOGEN's AlgoChainPattern (<Algo>Process, factories, schedulers):

McCOFFEE 2.20 has no Linux build of libalgochainpattern. An algorithm is a dataflow component here, its handlers run per sample.

• A topic with * or {x} is not subscribed to in McCOFFEE (its

TopicType is built from sub-topics the user code declares).

• accept_multiconnection_port, sequences (lists) in McCOFFEE types,

attributes' accessors, bufferingpolicy headers.

• A periodic thread is not mapped to McCOFFEE's timers component.

Rust

--target mccoffee-rust writes PYCOGEN's components in Rust. The user picks what they run on:

archec gen main.arche --target mccoffee-rust --option runtime=threads -o out   # std threads, no dependency
archec gen main.arche --target mccoffee-rust --option runtime=tokio -o out     # tokio tasks
archec gen main.arche --target mccoffee-rust --option runtime=embassy -o out   # embassy: no_std, static, heapless
cd out/mccoffee-rust && cargo run -p mc-node1

What it writes, a Cargo workspace:

| Folder | What is in it | |---|---| | mcrt/ | the runtime: active objects, endpoints, pub-sub brokers, services, dataflow buffers, allocators, TCP | | model/ | the types and messages; each interface as a trait, with its calls, replies, errors, a dispatch and a proxy; the metrics' topics | | components/<c>/ | business.rs (the Core, yours), facets.rs (a facet per provided port), connections.rs (the receptacles), attributes.rs, container.rs (inputs, dispatch, lifecycle) | | bins/<binary>/ | one binary per process: allocators, active objects, instances, connections, lifecycle |

How the model maps:

• An interface is a trait. An out or inout argument is &mut T. A

oneway operation returns nothing and does not wait. One that raises returns Result<T, <Interface>Error>. An attribute is get_x (and set_x).

• A union is an enum. An exception is a struct in the error enum. A native

type is its rust name.

• A thread is an active object: a thread (or a task) and a bounded queue.

A call to an active part goes through its queue and waits for the reply. A passive or protected part is called in the caller's thread, under its mutex. A thread's period and step call the step each period.

• Patterns: client-server, request-reply (at once, or deferred to a

handler; the reply is what the handler sends on the port), fire-and-forget, pub-sub (a broker per process, topics with {field} parts), services (by name), dataflow (vectors with a header; push-threshold, threshold, history, max-processing; back-pressure is a full queue that waits; copy or share).

• Signal processing: an algorithm's initialize runs at init, configure

after connect, reset at release; a sink's window calls execute_preprocessing, execute and execute_postprocessing when they take nothing. A composite is flattened: its roles are instances, its delegations followed, its after the start order. A metric is published on /<part path>/<metric>.

• A link between two processes goes over TCP (threads, tokio), with a

hand-written serialization of the model's types (mcrt::Wire). A process listens on its listen option (127.0.0.1:7101… by default). On embassy, mcrt::Transport is a trait to implement.

• The bodies in the action language come from the model's rust

translation, in the Core's regions: send, port.op(…), now().

• Your code goes between PYCOGEN's markers

(// -- start of user code: Filter::Core::execute), and is kept.

Options: runtime, run-for (ms a process runs; 0: until mcrt::finish()), listen, dataflow-capacity (embassy), deny-warnings. A binary's options mccoffee-rust { run-for 300 } sets its own. mcrt::check(ok, "what") is a test's assertion: the process exits 1 when one failed.

Priorities and affinity: std and tokio have none (the model's are noted, thread-priority can set them). On embassy, a thread of higher priority would go on an InterruptExecutor on a chip (noted in main.rs).

For a chip, an embassy binary builds with its cortex-m feature (a stand-in time driver and memory.x to replace):

cargo build -p mc-node0 --target thumbv7em-none-eabihf --no-default-features --features cortex-m