path: root/README

mcpu-cpp
========

mcpu-cpp is the preprocessor/front-end source manager for MCPU programming
languages.  It is an independent component of the LibMPU/LibMPUIO/LibMCPU
software platform and provides preprocessing, language selection, include-path
management, dependency generation and preprocessing diagnostics for the MCPU
toolchain.

Text model
----------

External text files are UTF-8.  Internally mcpu-cpp uses strict UCS-2 through
LibMPUIO.  UTF-8 scalar values which cannot be represented by UCS-2 are
rejected.  Legacy external code pages are not supported; the external representation is UTF-8.

Lexical analysis
----------------

mcpu-cpp does not use flex.  Its preprocessing scanners are hand-written and
operate on the internal UCS-2 text model.  The `#if` expression grammar and its
expression lexer are generated by ZUBR 4.1.0.

Parser architecture
-------------------

The generated `src/mcpp-expr.c` is shipped in release archives, so ordinary
builds do not require ZUBR.  In the developer Git tree the generated parser
may be omitted; the root `./bootstrap` script regenerates it from
`src/mcpp-expr.zubr` with ZUBR 4.1.0 before regenerating the Autotools files.

Documentation
-------------

The normative preprocessing manual is maintained in two synchronized forms:

  doc/mcpu-cpp-en.md   English
  doc/mcpu-cpp-ru.md   Russian

Both documents describe the same public contract and are distributed together.

Configuration
-------------

mcpu-cpp reads UTF-8 configuration files using a simple `NAME = value;`
syntax.  Shell-style `$NAME` and `${NAME}` references are expanded from values
already defined in the effective configuration and then from the process
environment.

MCPU uses one relocatable installation tree rather than a versioned directory
per tool.  With `--prefix=/usr --libdir=/usr/lib64`, `make install` creates:

```text
/usr/lib64/mcpu/
  bin/mcpu-cpp
  etc/mcpu-cpp.conf
  include/
  lib/
```

`/usr/bin/mcpu-cpp` is a public symbolic link to
`../lib64/mcpu/bin/mcpu-cpp`.  The absolute `/usr/lib64/mcpu` path is an
install-time choice only; it is not embedded as the runtime MCPU root.

On Linux, mcpu-cpp resolves its real executable through `/proc/self/exe`, takes
the parent of the executable directory as the MCPU runtime root, and derives
`<root>/etc/mcpu-cpp.conf` and `<root>/include` from it.  A fallback based on
`argv[0]`, `PATH`, and `realpath(3)` is used only when `/proc/self/exe` cannot
be read.  Consequently a complete MCPU tree may be copied or moved without
rebuilding mcpu-cpp.

This is the intended ecosystem-wide layout for future `mcpu-as`, `mcpu-ld`,
`mcpu-run`, libraries and CRT components as well: tool versions do not define
separate roots; a coherent MCPU environment is identified by one physical
runtime tree.

Configuration layers are applied in this order:

```text
runtime-derived defaults
<runtime-root>/etc/mcpu-cpp.conf
/etc/mcpu/mcpu-cpp.conf                 optional
$HOME/.mcpu/etc/mcpu-cpp.conf           optional, highest priority
```

The packaged config deliberately does not store an absolute default system
include path.  Before reading any config file mcpu-cpp sets
`MCPU_CPP_SYSTEM_INCLUDE_PATH=<runtime-root>/include`; any higher-priority
configuration may replace that value or set it empty.

`--config-file FILE` reads only FILE on top of the runtime-derived defaults.
`--no-config` reads no configuration files at all but keeps those runtime
defaults.  `--sys-root=PATH` uses `PATH` as the MCPU root, sets the effective
system include tree to `PATH/include`, and implies `--no-config`.  `PATH` may
be absolute or relative; a relative path is resolved from the invocation
working directory.  Use `-nostdinc` when the effective standard-system include
tree itself must be suppressed for one invocation.

Installation does not create `/etc/mcpu`; that directory is reserved for an
optional distributor or system-administrator override.  The per-user
configuration is deliberately not versioned.

Recognized path variables are:

  MCPU_CPP_INCLUDE_PATH
  MCPU_CPP_DIFF_INCLUDE_PATH
  MCPU_CPP_DIFT_INCLUDE_PATH
  MCPU_CPP_ALG_INCLUDE_PATH
  MCPU_CPP_AS_INCLUDE_PATH
  MCPU_CPP_AVM_INCLUDE_PATH
  MCPU_CPP_ACS_INCLUDE_PATH
  MCPU_CPP_SYSTEM_INCLUDE_PATH
  MCPU_CPP_AFTER_INCLUDE_PATH

User and AFTER path lists use the host PATH separator (`:` on UNIX systems).
`MCPU_CPP_SYSTEM_INCLUDE_PATH` is different: it is one replaceable root of the
MCPU system-header tree.  Its runtime-derived default is
`<runtime-root>/include`.  When a switched language is active mcpu-cpp
searches `<root>/<lang>` and then `<root>`.  A higher-priority configuration can
replace the root completely for a developer/tester sandbox, or set it to an
empty value to disable the configured system tree.

Command line
------------

  mcpu-cpp [options] [input [output]]

Important options:

  -o FILE              write output to FILE instead of stdout
  -D NAME[=VALUE]      define a command-line macro
  -U NAME              undefine a command-line macro
  -imacros FILE        preprocess FILE for macro state; discard its output
  -include FILE        preprocess FILE before the primary input
  -I DIR, -IDIR        add a user include directory
  -isystem DIR         add an explicit system include directory
  -idirafter DIR       add a directory searched after system directories
  -nostdinc            suppress the effective standard-system include tree
  -dM                   dump non-predefined macros to stdout
  -dMP                  dump predefined macros first, then other macros
  -dD                   preserve #define directives in normal output
  -dconfig              dump effective configuration variables to stdout
  -dsearch-dirs         dump effective include search directories and exit
  -M                    output make dependencies including system headers
  -MM                   output make dependencies excluding system headers
  -MD                    write dependencies and keep preprocessing output
  -MMD                   like -MD but exclude system headers
  -MF FILE               write dependencies to FILE ('-' means stdout)
  -MT TARGET             set unquoted make dependency target
  -MQ TARGET             set make-quoted dependency target
  --object-suffix SFX    set object suffix used for dependency targets
  -w                     suppress all warnings
  -Wcomment[s]            warn about nested /* and multi-line // comments
  -Wno-comment[s]         disable comment warnings even under -Wall
  -Wall                   enable all optional warning classes
  -Werror                 promote every emitted warning to an error
  -Wno-error              keep emitted warnings as warnings
  --config-file FILE   use only FILE as the configuration file
  --no-config          do not read config files; keep runtime-derived defaults
  --sys-root=PATH      use PATH as MCPU root and do not read config files
  -v, --verbose        print configuration and include activity
  --help               print help
  --version            print version

Include search
--------------

For `#include "file"`, the physical directory containing the current source
file is searched first.  For `#include <file>`, that first step is omitted.
The remaining include search order is normative:

```text
explicit -I
explicit -isystem
MCPU_CPP_<LANG>_INCLUDE_PATH
MCPU_CPP_INCLUDE_PATH
MCPU_CPP_SYSTEM_INCLUDE_PATH/<lang>
MCPU_CPP_SYSTEM_INCLUDE_PATH
explicit -idirafter
MCPU_CPP_AFTER_INCLUDE_PATH
```

Command-line include directories therefore override persistent configuration.
The language-specific user paths are freely configurable.  The system path is
a single effective root; mcpu-cpp derives the fixed `<root>/<lang>` directory
itself, so there are intentionally no
`MCPU_CPP_SYSTEM_<LANG>_INCLUDE_PATH` variables.  Replacing
`MCPU_CPP_SYSTEM_INCLUDE_PATH` replaces the complete installed system-header
tree rather than adding another directory.  An empty effective value disables
the configured system tree.  Neither `-idirafter` nor
`MCPU_CPP_AFTER_INCLUDE_PATH` acquires automatic language subdirectories.

`#include_next` is intended for wrapper headers.  It remembers the exact
physical entry of this effective search chain that supplied the current header
and resumes at the following entry.  This allows an explicit wrapper to alter
policy and then continue into a sandbox or installed system tree without
copying the original header or hard-coding its absolute pathname.  The
`"file"` and `<file>` forms are equivalent for `#include_next`, and logical
names established by `#line` do not affect physical search provenance.

`#pragma once` marks the current physical file as processed for the remainder
of the preprocessing run.  Identity is the filesystem device/inode pair, not
the pathname spelling, so the same file reached through another relative name,
a symbolic link or a hard link is skipped.  The exact active directive is
consumed by mcpu-cpp; an inactive `#pragma once` has no effect.  Other pragmas
remain in output for later compiler stages.

Command-line forced files
-------------------------

`-imacros FILE` and `-include FILE` use one deterministic preprocessing
pipeline.  Predefined macros are installed first, all command-line `-D`/`-U`
actions are then applied in their own command-line order, every `-imacros`
file is processed in command-line order, every `-include` file is processed in
command-line order, and only then does preprocessing enter the primary input.
The relative placement of `-imacros` and `-include` options in argv therefore
does not interleave the two classes: every `-imacros` always precedes every
`-include`.

An `-imacros` file goes through the ordinary preprocessing machinery, including
`#include`, macro definition/undefinition, conditional directives, `#lang`,
`#pragma once`, diagnostics and dependency tracking.  Its normal preprocessing
output, including output from headers reached from that file, is discarded.
The resulting macro table and other preprocessing state remain available to
subsequent forced files and to the primary input.  `-include` uses the same
preprocessing machinery but retains its normal output, as if the header had
been included immediately before the primary source.

The operand of either forced-file option has GNU-style command-line search
semantics.  An absolute path is used directly.  A relative path is searched
first in the current working directory and then through the ordinary quoted
include chain shown above.  The physical directory of the primary input does
not receive the special first priority that a literal `#include "file"` inside
that source would receive.  Once a forced file has been found, quoted includes
inside it are resolved normally relative to that file's physical directory,
and `#include_next` retains the search-chain provenance of the entry that found
it.

Forced files are ordinary physical dependencies.  Files reached through CWD or
user include classes remain user dependencies; files found through `-isystem`,
the configured system tree, `-idirafter` or configured AFTER paths retain
system dependency class for `-MM`/`-MMD`.


Dependency generation
---------------------

`-M` preprocesses the translation unit and writes one make rule instead of
normal preprocessor output.  The rule contains the main source file and every
physical header actually reached through the ordinary `#include` /
`#include_next` pipeline, including system headers.  Repeated spellings,
symbolic links and hard links to the same physical file are recorded once.
Logical names established by `#line` are never dependency names.

`-MM` uses the same traversal but omits headers found through `-isystem`, the
configured MCPU system tree, `-idirafter` and configured AFTER paths, and also
omits headers reachable only as descendants of such a system header.  Quote
versus angle include spelling does not decide whether a dependency is system.
If the same physical file is also reached directly through a user include
context, it remains a user dependency.

The default target is the input basename with its suffix replaced by the
configured object suffix (normally `.o`).  In dependency-only `-M`/`-MM` mode,
`-MF FILE` selects the make-rule destination; without `-MF`, the existing
mcpu-cpp `-o FILE` destination remains available.

`-MD` and `-MMD` generate dependencies as a side effect and do **not** suppress
normal preprocessing output.  `-MD` includes system headers like `-M`; `-MMD`
applies the same user-only filter as `-MM`.  Without `-MF`, the dependency file
is `<input-basename>.d` in the current directory, or is derived from ordinary
`-o` output by replacing its suffix with `.d`.  `-MF FILE` overrides that
automatic name, and `-MF -` sends the dependency rule to stdout.

Options `-MT` and `-MQ` are intentionally left for the next dependency stage.

#lang contract
--------------

The initial language state is `0`.  It is the base MCPU language state and is
installed before preprocessing begins.  `0` is deliberately not a parameter of
`#lang`.

`#lang` requires exactly one quoted language name:

    #lang "diff"

The text inside the quotes is not escape-decoded.  It must be a non-empty
single word without whitespace, and the closing quote must occur on the same
physical source line.  After the closing quote only whitespace is allowed.
The accepted names come only from the internal language table: `diff`, `dift`,
`alg`, `as`, `avm`, and `ACS`.  Matching is ASCII case-insensitive, so for
example `"diff"`, `"Diff"`, `"DIFF"` and `"dIfF"` all select the same
language.  Unsupported names such as `"0"`, `"c"` and `"vasm"` are errors.

External whitespace is normalized when `#lang` is copied to output.  For
example:

        #   lang    "DiFf"      

is emitted as:

    #lang "DiFf"

The spelling inside the quotes is preserved as written.

`#lang` and `#endlang` form one translation-unit-wide stack.  The stack is not
reset at an `#include` boundary.  Therefore a `#lang` in one file and its
matching `#endlang` in another included file are intentionally legal, exactly
as required by the macro-expansion contract.

Both directives are passed through to output so the later language dispatcher
and parser layer can observe the same language boundaries.  `#lang` is emitted
in the normalized form described above.

Macro operators
---------------

Function-macro stringification (`#`) is supported.
The operator uses the raw, unexpanded actual argument, removes leading and
trailing whitespace, folds internal whitespace outside quoted tokens to one
space, and escapes double quotes and backslashes as required by the resulting
quoted string.  `##` token concatenation is also supported and rescans the concatenated token
through the ordinary macro-expansion path.

The completed predefined ABI/environment layer from 0.0.12 is preserved.
INT/UINT DECIMAL_DIG counts only decimal digits of the corresponding numeric
maximum, without a sign or terminating NUL.  Integer decimal precision and
Real DECIMAL_DIG/MANT_DIG metadata cover every configured type width;
large MAX/MIN/EPSILON textual values remain capped at 256 bits.

`__MCPU_CPP_VERSION__` is the mcpu-cpp package version.  `_ARCH_MCPU` identifies
the target.  The LibMPU profile macros use the MCPU namespace:
`__MCPU_MACHINE_REGISTER_WIDTH__`, `__MCPU_REAL_IO_LIMIT__` and
`__MCPU_MATH_FN_LIMIT__`.

MCPU pointers are always 64 bits independently of the host.  `intptr` and
`ptrdiff` are signed `int64`; `uintptr` is `uint64`:

  __INTPTR_TYPE__   int64
  __INTPTR_WIDTH__  64
  __INTPTR_MAX__    0x7fffffffffffffff
  __UINTPTR_TYPE__  uint64
  __UINTPTR_WIDTH__ 64
  __UINTPTR_MAX__   0xffffffffffffffff
  __PTRDIFF_TYPE__  int64
  __PTRDIFF_WIDTH__ 64
  __PTRDIFF_MAX__   0x7fffffffffffffff

The configured LibMPU size and signed-size types are exposed only in the MCPU
namespace.  For a 64-bit profile the public contract is:

  __MCPU_SIZE_TYPE__       uint64
  __MCPU_SIZE_WIDTH__      64
  __MCPU_SIZEOF_SIZE__     8
  __MCPU_SIZE_MAX__        0xffffffffffffffff
  __MCPU_SSIZE_TYPE__      int64
  __MCPU_SSIZE_WIDTH__     64
  __MCPU_SIZEOF_SSIZE__    8
  __MCPU_SSIZE_MAX__       0x7fffffffffffffff

Integer TYPE/WIDTH/SIZEOF metadata is generated for every power-of-two LibMPU
integer family through `NB_I_MAX * 8`.  Real and Complex TYPE/WIDTH/SIZEOF
metadata is generated through the configured `MPU_REAL_IO_LIMIT`.  Complex
WIDTH is the language type parameter: `complex128` has WIDTH 128 but SIZEOF 32
bytes, because it stores two real128 components.

Every available Real family also exports two compact conversion/layout
properties through `MPU_REAL_IO_LIMIT`: `__SIZEOF_REAL<bits>_EXP__` comes from
LibMPU `_sizeof_exp()`, while `__REAL<bits>_MAX_STRLEN__` comes from
`_real_max_string()`.  MAX_STRLEN is a number of characters, not bytes; a
zero-terminated `char8` or `char16` buffer therefore needs at least
`MAX_STRLEN + 1` elements.

Large numeric textual values remain deliberately capped at 256 bits.  Integer
MAX and Real MAX/MIN/EPSILON/exponent-value macros are not emitted above that
width.  Integer DECIMAL_DIG and Real DECIMAL_DIG/MANT_DIG remain available for
the complete configured families.  Integer MIN expressions are never
predefined.  This keeps `-dMP` compact while preserving useful precision,
structural and text-buffer metadata for large LibMPU types.

The future language uses fixed-width names.  Character types are `char8` and
`char16`; their sizes are `__SIZEOF_CHAR8__` and `__SIZEOF_CHAR16__`.  Ordinary
C `char`, `short`, `int`, `long`, `wchar_t`, and the C-style `char*_t` names are
not part of the target language type model.

The effective `MCPU_CPP_SYSTEM_INCLUDE_PATH` automatically contributes a
language-specific subdirectory for each active switched language.  For example,
with the runtime-derived root and `#lang "diff"`,
`<runtime-root>/include/diff` is searched before
`<runtime-root>/include`.  These derived directories need not exist and
require no extra configuration variables.

`-dM` dumps only the current non-predefined macro table in deterministic
`#define` form.  `-dMP` first dumps active predefined macros and then the
non-predefined macros; each group is sorted by name.
`-dconfig` dumps the effective configuration variables in sorted `NAME = value;`
form.  `-v` prints the include-related effective configuration values once,
after all configuration layers have been resolved, and then preserves the usual
include/language runtime trace.  `-dsearch-dirs` prints `search: DIR` lines for
the effective global include directories in semantic priority order and exits
without preprocessing.  The dynamic source-directory step used by quoted
includes is not part of that global dump.  None of these dump actions requires
an input file.

The dynamic source macros `__FILE__`, `__LINE__`, `__BASE_FILE__`,
`__INCLUDE_LEVEL__`, `__DATE__` and `__TIME__` use one translation-unit
source stack and timestamp.  Stringification, concatenation and conditional
compilation are part of the current preprocessing contract.

Parser generation
-----------------

The #if expression parser is generated by ZUBR 4.1.0 from
`src/mcpp-expr.zubr`.  The grammar also contains the UCS-2 lexical analyzer.
The generated source `src/mcpp-expr.c` is included in release archives, so a
normal build from a release archive does not require ZUBR.

The conditional-expression evaluator is deliberately 64-bit only.  Integer
literals may use `U`/`u`, or the MCPU width suffix `zNNN[Uu]` / `ZNNN[Uu]`.
For valid widths up to 64, the low N bits are taken and then sign-extended
(`zNNN`) or zero-extended (`zNNNu`) to 64 bits; all later operations remain
64-bit and the original width is forgotten.  Widths above 64 are rejected in
conditional directives.  Invalid widths not exceeding 64 produce a warning
and the width suffix is ignored.  C `L`/`LL` integer suffixes are not accepted.

`#error` and `#warning` are implemented as diagnostic directives.  Their
arguments are not macro-expanded.  Outside quoted tokens, whitespace sequences
are folded to one space for the diagnostic text.  `#error` stops preprocessing;
`#warning` continues.  Both directives disappear from normal and `-dD` output,
and both are ignored in inactive conditional branches.

Warning control follows a compact GNU-like model.  `-Wcomment` and `-Wcomments`
enable warnings for `/*` inside an existing block comment and for a
backslash-newline continuing a `//` comment; `-Wall` currently enables this
optional warning class.  The specific `-Wno-comment`/`-Wno-comments` setting
overrides `-Wall` regardless of command-line order.  `-w` globally suppresses
all warnings, including `#warning`, invalid `zNNN` widths, macro redefinition,
invalid `##` paste and enabled comment diagnostics.  It does not suppress
errors.  `-Werror` promotes every warning that is actually emitted to an error
and unsuccessful preprocessing; because `-w` prevents warning emission first,
`-w -Werror` and `-Werror -w` are equivalent and successful when no independent
error occurs.  Likewise, warning classes may be enabled by `-Wcomment` or
`-Wall`, but remain silent under `-w` regardless of option order.  `-Wno-error`
restores ordinary warning severity.  `-Werror` does not by itself enable
optional warning classes.

When `mcpp-expr.zubr` is changed, the ordinary Automake `.zubr.c` rule
regenerates the C source with:

  zubr -vl -s -Bmcpp_ -o mcpp-expr.c mcpp-expr.zubr


Developer bootstrap
-------------------

A Git checkout may omit files that are regenerated mechanically, including
`configure`, `Makefile.in`, Automake helper scripts, `config.h.in`, `aclocal.m4`
and `src/mcpp-expr.c`.  Regenerate them with:

```text
./bootstrap
```

`./bootstrap --target-dest-dir=DIR` follows the LibMPU/LibMPUIO convention for
using Autoconf macro/header directories from a target ROOTFS.  Release archives
remain self-contained and do not require bootstrap before `configure`.

Build
-----

mcpu-cpp uses Autoconf/Automake and obtains LibMPUIO compilation and linker
flags from `mpuio-config`, following the conventions of LibMPU and LibMPUIO.
No pkg-config or flex dependency is introduced.  The 1.0.3 release is
developed and tested against LibMPU 1.0.25 and LibMPUIO 1.0.4.  The LibMPUIO
UCS-2 ctype API is required.  ZUBR 4.1.0 is the parser-regeneration tool; it is
not required for a normal build from a release archive containing the generated
`src/mcpp-expr.c`.

A normal build is:

  ./configure --prefix=/usr --libdir=/usr/lib64
  make
  make tests
  make install