Documentation overview

The Diátaxis framework is a way to classify documentation content into four groups with distinct use cases.

Documentation exists along two axis: study or work; pracital steps and theory. Practical learning happens in tutorials, and they are backed up by conceptual background information (theory). On the work oriented side, documentation should contain exhaustive reference information (every commandline flag, every yaml property, their type and whether they are optional or not etc.) as well as narrow, task oriented guides.

At Stackable we have to document the platform as a whole, individual operators, products and command line tools like stackablectl.

Conceptual information lives at the platform level in the Concepts section. Some operators have their own specific concepts such as ZNodes for ZooKeeper, but most concepts apply to all operators (i.e. roles and role groups, resource management, config overrides).

Tutorials (learning oriented guides) exist at the top level but there are also the Getting Started guides for all the individual operators. Tutorials provide complete set up information and require very little to be there already in contrast to guides (work instead of study oriented) which are more focused on solving specific tasks. There are also the Demos which are complete solutions that showcase how the platform integrates different products in a unified, reproducible and transparent way.

Guides are also instructional, but focus a narrow topic, something that the user might want to solve in their particular setup. For example: "How do I set up Kerberos with HDFS?" or "How do I connect Superset to Druid?" The guide has to account for different use-cases (i.e. the user is using their own pre-existing ZooKeeper instead of our ZooKeeper). This is not the case for tutorials. Since this kind of information is typically product specific, it is located in the usage guide section of individual operators.

Reference information for the Stackable platform entails all the settings and Options in our YAMLs, which we generate. The reference is found at https://crds.stackable.tech/ and generated from the crddocs repository.

Antora uses a playbook to build the documentation. It pulls documentation content from all the individual operator repositories, so an operators documentation is maintained in the same repository as the code. Antora builds a static website which we serve over Netlify. The web template of the documentation is also custom made and is developed in the documentation-ui repository.

For search, we use pagefind - a static search. The search index is generated as part of the build process and no external index is queried during search.

For Antora, the Antora Zulip chatroom is a good place to get help (besides the documentation)!

Building the documentation and also the deployment process on Netlify are documented in the README file of the documentation repository.

The getting started guides for each operator are backed by an executable script from which the documentation includes excerpts. This has the benefit that the guides are easily tested. When writing a new guide, please also write it in this way. Have a look at the existing getting started guides on how to do this.

There is a templating mechanism in the docs. This has been introduced to template in mostly version numbers, so the updating doesn’t have to be done by hand.

Every Operator repo has a script scripts/docs_templating.sh and a file with templating variables docs/templating_vars.yaml. The script applies the variables to all .j2 files in the docs directory.

This is used for getting started scripts, and in there only for versions of operators and for Helm Chart URLs.

Without this templating mechanism, every release these values would need to be updated by hand (or possibly with a search and replace) with is error prone and time consuming.

The documentation consists of two components, the home and management component. All documentation for the operators is in the home component, and it is versioned with the platform versions (23.11, 24.3, 24.7 etc.). The management component contains docs for stackablectl and the Stackable cockpit; it is not versioned (there is only a single, current version).

The home component is actually a distributed component, which means that it is split across multiple repositories. Each operator repository has a docs directory where the docs are found, and in there the antora.yml file specifies that the component is home, which means that the home component is partially defined across all operator repositories.

For versions, all Stackable repositories use trunk based development, and so the documentation is also pulled from different release branches in each repository. Each branch contains only a single version, and the main branch contains the nightly docs version. Using branches to structure component version is also recommended by Antora.