#20 map/deploy every deployable folders

Reviewed-on: #25
Reviewed-by: welwei_tilman <tilman.sattler@stud.th-rosenheim.de>

.gitignore

@@ -0,0 +1,3 @@
+deploy
+sstate-cache
+downloads

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) [year] [fullname]
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

NOTICE

@@ -0,0 +1,3 @@
+Used docker container:  MIT license, copyright notice: (C) 2021-2023 jhnc-oss
+./dev/bootstrap.sh (modified):     MIT license, copyright notice: (C) 2021-2023 jhnc-oss
+./dev/init_env.sh (modified):      MIT license, copyright notice: (C) 2021-2023 jhnc-oss

README.md

@@ -1,6 +1,148 @@
-# CAROS
-# _C_ar _R_ealtime _O_perating _S_ystem.
+# CaROS
+# Car Realtime Operating System.
 
 A yocto base distro for DIY car radio
 
-the CaROS specific software is found in meta-caros
+The CaROS specific software is found in meta-caros
+
+
+## Description
+
+
+## Setup
+
+### Preliminaries
+
+Install required packages:
+
+- podman
+
+### Get yocto-build
+```shell
+git clone git@git.larsniesen.de:caros/caros.git
+```
+
+### User Namespace Mapping
+
+Ensure, that /etc/subgid and /etc/subuid contain corresponding
+and sane entries for your account (podman run user).
+If not you might need to run the following setup script (requires restart):
+```shell
+$ sudo ./dev/AddUserToSubgidAndSubuid.sh
+```
+
+## Start development build container
+
+```shell
+$ ./dev/bootstrap.sh [--deploy] [--sstate]
+```
+
+By adding the parameter --deploy, bootstraph.sh maps a deploy folder into the build container.
+Built RPMs will be placed in this folder.
+
+The parameter --sstate maps the build cache directory. This allows the cache to be preserved when the container
+is closed thus significantly reducing build time.
+
+You will end up in the yocto build container as user yocto e.g.:
+
+```shell
+[yocto@1453b5f0ab01 build]$
+```
+
+
+
+## Collaboration
+
+
+
+### Working with branches
+
+Every contributor should use his/her initials (eg. Max Mustermann uses 'mm') as the branch prefix.
+After the prefix comes the issue ID and a short name describing the work.
+
+The branch of Max Mustermann regarding the issue "#13 Integrate Fancy UI into the new web server"
+should be named:
+
+```
+mm/13-integrate-new-web-ui
+```
+
+### Pull requests
+
+The pull request is to be opened **after** testing the branch feature.
+The exception to this are draft PRs
+
+Before merging each PR needs to be reviewed from at least one other contributor
+
+### Merging
+
+Merges are to be done the following:
+
+1. Update main.
+```
+$ git pull origin
+```
+2. Rebase main onto your feature branches.
+    In your feature branch do:
+
+```
+$ git rebase origin/main
+```
+
+3. Test Again if the rebase broke something.
+4. Squash your commits into one.
+```
+$ git rebase -i HEAD~<num of commits>
+```
+In the now open editor replace `pick` with `squash` on all of your commits except the first.
+After that quit the editor and another editor should pop up. Now enter the new commit message.
+The commit message should look like this:
+```
+<issue id>: Description
+```
+For the above example it would be:
+```
+13: Integrate the new web-UI to improve the experience for new users.
+```
+The Description should be **why** the commit was made and not **what** the commit does.
+
+5. Now we add the hash at the start of the commit with:
+```
+git commit --amend -m "#13: Integrate the new web-UI to improve the experience for new users."
+```
+The amend flag lets you _change_ your last commit message. You have to enter your complete commit message but with a hash at the start.
+
+6. Now force push your branch to the remote.
+
+```
+$ git push --force
+```
+7. Fast-forward merge on GitLab without merge commit.
+
+
+For those who are not comfortable with git rebase please follow these steps.
+It is still recommended to do the squashing on the terminal.
+
+
+1. Update main.
+```
+$ git pull origin/main
+```
+
+2. Rebase main onto your feature branches.
+    In your feature branch do:
+
+```
+$ git rebase origin/main
+```
+
+3. Test Again if the rebase broke something.
+
+4. Now force push your branch to the remote.
+
+```
+$ git push --force
+```
+
+5. In the PR now fast-forward merge with squashing and without merge commit.
+

Roadmap.md

@@ -0,0 +1,69 @@
+First ideas and Roadmap
+=======================
+
+# Planned functional features
+
+- Music (files & bluetooth & aux) incl. visuializer
+- Google-Maps etc.
+- NFS / SMB / HTTP server for manuals etc.
+- WIFI AP support
+- Hardened distro
+- Linux-RT Kernel for CAN-bus
+- CAN-bus
+
+
+
+# UI/UX features
+
+- Hacky UI (no plymouth, terminal tools, etc.)
+
+
+# Planned software features
+
+- QT6
+- Systemd
+- Wayland
+- Linux-RT
+- ...
+
+# Hardware considerations
+
+- x86_64
+    - Intel nuc
+    - China Laptop
+- 3D printed parts for mounting (opensource CAD files, preferred STP)
+- Nav-Buttons/Rotary encoders (navigation besides Touchscreen)
+- Touchscreen monitor
+- USB hub (inside the glove compartment, on the side of the screen)
+
+# Misc
+- Cava for FFT and music visuialisation
+
+
+
+# Strength of the Team members
+
+Albert: ??? Frontend?
+
+David:  Hardware, car stuff & co.
+
+Emre:   Embedded (OS / Yocto). Frontend
+
+Lars:   Embedded (Kernel + driver). Yocto.
+
+Tilman: Yocto (beginner). Deamons (system layer). QT6
+
+# First timeline and work distribution idea
+
+1. Minimal image (Hello world OS)            - Lars, Tilman, (Emre)
+2. Hardware discussion
+3. Deamon Setup + Hardware                  - (Lars), Emre, Tilman, David
+    - Musik
+    - GPIO
+4. UI                                       - ?
+    - Visuializer
+    - Settings
+5. NFS Server & Config of OS. - (Lars), Emre, Tilman, David
+6. Can bus / Relatime - Lars, Emre, Tilman, David
+
+__Notes:__ Points 3, 4 and 5 are workable in parallel.

default.xml

@@ -0,0 +1,32 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<manifest>
+
+  <!-- settings -->
+  <default sync-j="4" />
+
+  <!-- remotes -->
+  <remote name="caros" fetch="https://git.larsniesen.de/caros" />
+
+  <!-- layers -->
+  <project remote="caros"
+           name="poky"
+           path="poky"
+           revision="kirkstone" />
+
+
+  <project remote="caros"
+           name="meta-openembedded"
+           path="meta-oe"
+           revision="kirkstone" />
+
+  <project remote="caros"
+           name="meta-qt6"
+           path="meta-qt6"
+           revision="lts-6.2.9" />
+
+  <project remote="caros"
+           name="meta-selinux"
+           path="meta-selinux"
+           revision="kirkstone" />
+</manifest>
+

dev/AddUserToSubgidAndSubuid.sh

@@ -0,0 +1,8 @@
+#!/bin/sh
+
+set -o errexit
+set -o nounset
+
+USERNAME="$(logname)"
+echo "${USERNAME}":"$(id -u "${USERNAME}")":65536 >> /etc/subuid
+echo "${USERNAME}":"$(id -g "${USERNAME}")":65536 >> /etc/subgid

dev/bootstrap.sh

@@ -0,0 +1,90 @@
+#!/usr/bin/env bash
+
+set -o errexit
+set -o pipefail
+
+MANIFEST_BRANCH="${1:-main}"
+
+YOCTO_GID="4040"
+YOCTO_UID="2000"
+YOCTO_USER="yocto"
+YOCTO_WORKDIR="/opt/${YOCTO_USER}"
+
+[[ -d "$PWD"/downloads ]] || mkdir "$PWD"/downloads
+[[ -d "$PWD"/sstate-cache ]] || mkdir "$PWD"/sstate-cache
+
+
+
+subgidSize=$(( $(podman info --format "{{ range .Host.IDMappings.GIDMap }}+{{.Size }}{{end }}" ) - 1 ))
+subuidSize=$(( $(podman info --format "{{ range .Host.IDMappings.UIDMap }}+{{.Size }}{{end }}" ) - 1 ))
+
+
+if [ -z "$DO_DEPLOY" ] ; then DO_DEPLOY=false ; fi
+if [ -z "$DO_SSTATE" ] ; then DO_SSTATE=false ; fi
+
+usage() {
+    echo " --deploy         map deploy folder for outside access"
+    echo " --sstate         map sstate folder to preserve regarless of container"
+    echo " --name=<name>    assign a name to the container"
+}
+
+while [ -n "$1" ] ; do
+    case "$1" in
+        --deploy)
+            DO_DEPLOY=true
+            ;;
+        --sstate)
+            DO_SSTATE=true
+            ;;
+        --name=*)
+            CONTAINER_NAME="$1"
+            ;;
+        *)
+            usage
+            exit 1
+            ;;
+    esac
+    shift
+done
+
+
+if $DO_SSTATE ; then
+    SSTATE_PATH="sstate-cache"
+    mkdir -p $SSTATE_PATH
+    sstate_param1="-v"
+    sstate_param2="${PWD}"/"${SSTATE_PATH}":"${YOCTO_WORKDIR}"/"${SSTATE_PATH}"
+fi
+
+if $DO_DEPLOY ; then
+    DEPLOY_PATH="deploy"
+    mkdir -p $DEPLOY_PATH
+    rm -rf "$DEPLOY_PATH"/*
+    deploy_param1="-v"
+    deploy_param2="${PWD}"/"${DEPLOY_PATH}":"${YOCTO_WORKDIR}"/"${DEPLOY_PATH}"
+fi
+
+
+
+podman run \
+  -ti \
+  --rm \
+  --pull=always \
+  --gidmap ${YOCTO_GID}:0:1 \
+  --gidmap $((YOCTO_GID+1)):$((YOCTO_GID+1)):$((subgidSize-YOCTO_GID)) \
+  --gidmap 0:1:${YOCTO_GID} \
+  --uidmap ${YOCTO_UID}:0:1 \
+  --uidmap $((YOCTO_UID+1)):$((YOCTO_UID+1)):$((subuidSize-YOCTO_UID)) \
+  --uidmap 0:1:${YOCTO_UID} \
+  --user "${YOCTO_USER}:${YOCTO_USER}" \
+  --workdir "${YOCTO_WORKDIR}" \
+  ${deploy_param1} ${deploy_param2} \
+  ${sstate_param1} ${sstate_param2} \
+  -v "${PWD}"/default.xml:"${YOCTO_WORKDIR}"/default.xml \
+  -v "${PWD}"/meta-caros:"${YOCTO_WORKDIR}"/meta-caros:Z \
+  -v "${PWD}"/dev:"${YOCTO_WORKDIR}"/dev:Z \
+  -v "${PWD}"/downloads:"${YOCTO_WORKDIR}"/downloads:Z \
+  --env TEMPLATECONF="${YOCTO_WORKDIR}"/meta-caros/conf/templates \
+  --env SSTATE_PATH="${SSTATE_PATH}" \
+  ${CONTAINER_NAME} \
+  ghcr.io/jhnc-oss/yocto-image/yocto:37 \
+  bash -c "dev/init_env.sh "

dev/init_env.sh

@@ -0,0 +1,20 @@
+#!/usr/bin/env bash
+
+set -o errexit
+
+# Workaround for unsupported local manifest
+mkdir _local
+cd "${YOCTO_DIR}/_local"
+cp "${YOCTO_DIR}/default.xml" .
+git init -b master
+git add .
+git commit -m "local manifest"
+cd - > /dev/null
+
+repo --color=always init _local
+repo sync --no-clone-bundle
+rm -rf "${YOCTO_DIR}/_local"
+
+source poky/oe-init-build-env
+
+exec /bin/bash

meta-caros/conf/distro/caros.conf

@@ -0,0 +1,18 @@
+include conf/distro/include/preferred-versions.conf
+
+include conf/distro/poky.conf
+
+DISTRO = "caros"
+DISTRO_NAME = "CaROS (Car Realtime Operating System)"
+
+DISTRO_VERSION = "0.0.1"
+DISTRO_CODENAME = "ganymed"
+SDK_VENDOR = "-carossdk"
+SDK_VERSION = "${@d.getVar('DISTRO_VERSION').replace('snapshot-${METADATA_REVISION}', 'snapshot')}"
+SDK_VERSION[vardepvalue] = "${SDK_VERSION}"
+
+MAINTAINER = "Lars Niesen <lars.niesen@gmx.de>"
+
+TARGET_VENDOR = "-caros"
+
+DISTRO_FEATURES ?= ""

meta-caros/conf/distro/include/preferred-versions.conf

@@ -0,0 +1,5 @@
+#########################
+# Enforce package version
+#########################
+PREFERRED_VERSION_linux-yocto ?= "5.15%"
+PREFERRED_VERSION_linux-yocto-rt ?= "5.15%"

meta-caros/conf/layer.conf

@@ -0,0 +1,20 @@
+# We have a conf and classes directory, add to BBPATH
+BBPATH =. "${LAYERDIR}:"
+
+# We have recipes-* directories, add to BBFILES
+BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
+            ${LAYERDIR}/recipes-*/*/*.bbappend"
+
+BBFILE_COLLECTIONS += "meta-caros"
+BBFILE_PATTERN_meta-caros = "^${LAYERDIR}/"
+BBFILE_PRIORITY_meta-caros = "5"
+
+LAYERSERIES_COMPAT_meta-caros = "kirkstone"
+
+# This should only be incremented on significant changes that will
+# cause compatibility issues with other layers
+LAYERVERSION_meta-caros = "1"
+
+LAYERDEPENDS_meta-caros = "core"
+
+REQUIRED_POKY_BBLAYERS_CONF_VERSION = "2"

meta-caros/conf/templates/bblayers.conf.sample

@@ -0,0 +1,26 @@
+# POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
+# changes incompatibly
+POKY_BBLAYERS_CONF_VERSION = "1"
+
+BBPATH = "${TOPDIR}"
+BBFILES ?= ""
+
+BBLAYERS ?= " \
+    ${TOPDIR}/../poky/meta \
+    ${TOPDIR}/../poky/meta-poky \
+    ${TOPDIR}/../poky/meta-yocto-bsp \
+    ${TOPDIR}/../meta-qt6 \
+    ${TOPDIR}/../meta-selinux \
+    ${TOPDIR}/../meta-oe/meta-filesystems \
+    ${TOPDIR}/../meta-oe/meta-gnome \
+    ${TOPDIR}/../meta-oe/meta-initramfs \
+    ${TOPDIR}/../meta-oe/meta-multimedia \
+    ${TOPDIR}/../meta-oe/meta-networking \
+    ${TOPDIR}/../meta-oe/meta-oe \
+    ${TOPDIR}/../meta-oe/meta-perl \
+    ${TOPDIR}/../meta-oe/meta-python \
+    ${TOPDIR}/../meta-oe/meta-webserver \
+    ${TOPDIR}/../meta-oe/meta-xfce \
+    ${TOPDIR}/../meta-caros \
+"
+

meta-caros/conf/templates/conf-notes.txt

@@ -0,0 +1,27 @@
+
+ ██████╗ █████╗ ██████╗  ██████╗ ███████╗
+██╔════╝██╔══██╗██╔══██╗██╔═══██╗██╔════╝
+██║     ███████║██████╔╝██║   ██║███████╗
+██║     ██╔══██║██╔══██╗██║   ██║╚════██║
+╚██████╗██║  ██║██║  ██║╚██████╔╝███████║
+ ╚═════╝╚═╝  ╚═╝╚═╝  ╚═╝ ╚═════╝ ╚══════╝
+
+
+Welcome to the CaROS yocto buildsystem.
+
+
+You can now run 'bitbake <target>'
+
+Common targets are:
+    core-image-minimal
+    core-image-full-cmdline
+    core-image-sato
+    core-image-weston
+    meta-toolchain
+    meta-ide-support
+
+Other commonly useful commands are:
+ - 'devtool' and 'recipetool' handle common recipe tasks
+ - 'bitbake-layers' handles common layer tasks
+ - 'oe-pkgdata-util' handles common target package tasks
+

meta-caros/conf/templates/local.conf.sample

@@ -0,0 +1,121 @@
+MACHINE ?= "genericx86-64"
+
+DL_DIR ?= "${TOPDIR}/../downloads"
+
+SSTATE_DIR ?= "${TOPDIR}/../sstate-cache"
+
+DISTRO ?= "caros"
+
+PACKAGE_CLASSES ?= "package_rpm"
+
+#
+# SDK target architecture
+#
+# This variable specifies the architecture to build SDK items for and means
+# you can build the SDK packages for architectures other than the machine you are
+# running the build on (i.e. building i686 packages on an x86_64 host).
+# Supported values are i686, x86_64, aarch64
+SDKMACHINE ?= "x86_64"
+
+#
+# Extra image configuration defaults
+#
+# The EXTRA_IMAGE_FEATURES variable allows extra packages to be added to the generated
+# images. Some of these options are added to certain image types automatically. The
+# variable can contain the following options:
+#  "dbg-pkgs"       - add -dbg packages for all installed packages
+#                     (adds symbol information for debugging/profiling)
+#  "src-pkgs"       - add -src packages for all installed packages
+#                     (adds source code for debugging)
+#  "dev-pkgs"       - add -dev packages for all installed packages
+#                     (useful if you want to develop against libs in the image)
+#  "ptest-pkgs"     - add -ptest packages for all ptest-enabled packages
+#                     (useful if you want to run the package test suites)
+#  "tools-sdk"      - add development tools (gcc, make, pkgconfig etc.)
+#  "tools-debug"    - add debugging tools (gdb, strace)
+#  "eclipse-debug"  - add Eclipse remote debugging support
+#  "tools-profile"  - add profiling tools (oprofile, lttng, valgrind)
+#  "tools-testapps" - add useful testing tools (ts_print, aplay, arecord etc.)
+#  "debug-tweaks"   - make an image suitable for development
+#                     e.g. ssh root access has a blank password
+# There are other application targets that can be used here too, see
+# meta/classes/image.bbclass and meta/classes/core-image.bbclass for more details.
+# We default to enabling the debugging tweaks.
+EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
+
+#
+# Additional image features
+#
+# The following is a list of additional classes to use when building images which
+# enable extra features. Some available options which can be included in this variable
+# are:
+#   - 'buildstats' collect build statistics
+USER_CLASSES ?= "buildstats"
+
+PATCHRESOLVE = "noop"
+
+#
+# Disk Space Monitoring during the build
+#
+# Monitor the disk space during the build. If there is less that 1GB of space or less
+# than 100K inodes in any key build location (TMPDIR, DL_DIR, SSTATE_DIR), gracefully
+# shutdown the build. If there is less than 100MB or 1K inodes, perform a hard halt
+# of the build. The reason for this is that running completely out of space can corrupt
+# files and damages the build in ways which may not be easily recoverable.
+# It's necessary to monitor /tmp, if there is no space left the build will fail
+# with very exotic errors.
+BB_DISKMON_DIRS ??= "\
+    STOPTASKS,${TMPDIR},1G,100K \
+    STOPTASKS,${DL_DIR},1G,100K \
+    STOPTASKS,${SSTATE_DIR},1G,100K \
+    STOPTASKS,/tmp,100M,100K \
+    HALT,${TMPDIR},100M,1K \
+    HALT,${DL_DIR},100M,1K \
+    HALT,${SSTATE_DIR},100M,1K \
+    HALT,/tmp,10M,1K"
+
+#
+# Yocto Project SState Mirror
+#
+# The Yocto Project has prebuilt artefacts available for its releases, you can enable
+# use of these by uncommenting the following lines. This will mean the build uses
+# the network to check for artefacts at the start of builds, which does slow it down
+# equally, it will also speed up the builds by not having to build things if they are
+# present in the cache. It assumes you can download something faster than you can build it
+# which will depend on your network.
+# Note: For this to work you also need hash-equivalence passthrough to the matching server
+#
+#BB_HASHSERVE_UPSTREAM = "hashserv.yocto.io:8687"
+#SSTATE_MIRRORS ?= "file://.* http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"
+
+
+#
+# Hash Equivalence
+#
+# Enable support for automatically running a local hash equivalence server and
+# instruct bitbake to use a hash equivalence aware signature generator. Hash
+# equivalence improves reuse of sstate by detecting when a given sstate
+# artifact can be reused as equivalent, even if the current task hash doesn't
+# match the one that generated the artifact.
+#
+# A shared hash equivalent server can be set with "<HOSTNAME>:<PORT>" format
+#
+#BB_HASHSERVE = "auto"
+#BB_SIGNATURE_HANDLER = "OEEquivHash"
+
+#
+# Memory Resident Bitbake
+#
+# Bitbake's server component can stay in memory after the UI for the current command
+# has completed. This means subsequent commands can run faster since there is no need
+# for bitbake to reload cache files and so on. Number is in seconds, after which the
+# server will shut down.
+#
+#BB_SERVER_TIMEOUT = "60"
+
+# CONF_VERSION is increased each time build/conf/ changes incompatibly and is used to
+# track the version of this file when it was generated. This can safely be ignored if
+# this doesn't mean anything to you.
+CONF_VERSION = "2"
+
+DEPLOY_DIR = "${TOPDIR}/../deploy"