ll-killer-go layer: reuse build results and manage disk space

This article is reproduced from community contributor System233 and the project linglong-killer-self-service ， Original link>>>

ll-killer-go – layer subcommand: The final piece of the puzzle in the Linyaps app building revolution

Preface

Latest Updates in ll-killer-go v1.4.x version, a new layer subcommand, which is independent of ll-builder In addition to implementing the functions of building, packaging, mounting/uninstalling and parsing Linyapslayer files, no need to generate ~/.cache/linglong-builder The build cache wastes storage space and does not produce a large number of useless file copies, which greatly enhances the building experience of Linyaps applications.

introduction

In the packaging process of Linyaps applications, the traditional ll-builder tool has serious drawbacks due to its repetitive data storage, inefficient disk operations, and inflexible support for the build environment.

In comparison, ll-killer in layer The subcommand tool implements the build and packaging process in a more efficient, lightweight and flexible way, especially in layer Shows obvious advantages in file management. The shortcomings of

This article will introduce in detail ll-builder and emphatically elaborate on ll-killer layer The core advantages of subcommands, download and installation steps, and practical use cases of commonly used subcommands.

Related content : About using ll-kiler For detailed information on application packaging, please see the previous article:

ll-killer-go: New Linyaps application local construction system, with Ubuntu source GIMP migration example - Forum - Deepin Technology

Table of contents

1. Limitations and trade-offs of ll-builder
1.Storage waste is serious
2. Disk wear is large
3. Execute strip by default
4. Multi-task parallel construction problem
5. The build content is cleared frequently
6. Tool scalability is poor
7. The build environment is inconsistent with the runtime environment
8. Non-configurable compression algorithm
2. ll-killer construction environment and its advantages
1. Build environment consistency
2. The packaging process is efficient
3. Disk space and cache management optimization
4. Improved speed and flexibility
5. Reliability enhancement
3. Use cases of ll-killer layer subcommand
1. Get ll-killer
2. The packaging folder is layer
3. Efficiently build Linyaps projects and generate layers
4. Mount and uninstall layer
5. Output layer information
4. Summary

---

1. Limitations and trade-offs of ll-builder

ll-builder has serious flaws in its design, mainly including:

1. Severe storage waste

Every application built using ll-builder will ~/.cache/linglong-builder and linglong/output produces two copies. Even if you optimize to eliminate duplicate develop files, you will still end up with two duplicate binary copies. In the early days, up to four copies may even be generated. Although the ostree repository can hard link some duplicate files, it is still difficult to avoid the existence of two independent files under linglong/output.
Duplicate file storage:

~/.cache/linglong-builder
├── layers
│   ├── {base}
│   └── {app}       # binary+delelop模块 linglong/output的副本
├── merged
│   └── {app+base}  # 通过ostree硬链接叠加实现的副本
├── repo            # ostree仓库
└── states.json

.                   # 玲珑工程目录
├── linglong
│   ├── entry.sh
│   └── output
│       ├── _build
│       ├── binary  # binary模块 _build的副本
│       └── delelop # delelop模块 _build的副本
├── linglong.yaml
​

2. Disk wear

During the build process, the buildScript in the Linyaps project is written $PREFIX , ll-builder will be copied to linglong/output/binary and develop and then copy to ~/.cache/linglong-builder . During the packaging process, mkfs.erofs also needs to copy temporary files to add file headers. The entire process involves at least five file copies, resulting in amplification of read and write operations and increased disk burden.

Copy path:

1. build构建脚本->$PREFIX(linglong/output/_build)
2. linglong/output/{binary,develop}
3. ~/.cache/linglong-builder/{repo,binary,develop}
4. tmp.erofs
5. app.layer

• In older versions of Linyaps (approximately <=1.6.3), the read and write volume of steps 2 and 3 continued to double.

3. Execute strip by default

Enabled by default strip-symbols Perform symbol stripping on binary files, which may have unpredictable effects on applications and undermine user expectations for the files. ostree was introduced in

4. Multi-task parallel construction problem

Due to ~/.cache/linglong-builder , allowing only single-task writes; in addition, in the new version (v1.7.x), multi-task builds may cause the cache directory to be automatically renamed, causing cache corruption, ll-builder Unable to recover from errors on its own.

5. The build content is frequently cleared

Each build clears the contents of the previous build, making it difficult to adjust the output multiple times in the same build environment, accompanied by a large number of file copy operations.

6. Poor tool scalability

ll-builder only supports the built-in tool chain and cannot dynamically install new tools. If special runtime , and the documentation is insufficient, making it difficult to build.

7. The build environment and runtime environment are inconsistent

Forcibly introducing development dependencies during construction results in a deviation between the build environment and the final runtime environment, causing normal applications to be tested in the build environment to fail to run after being exported.

8. Non-configurable compression algorithm

In Linyaps 1.7.7 The layer is supported only when a higher version of erofs is introduced. zstd Compression algorithm, older versions of Linyaps cannot be installed and cannot detect and notify users of solutions, and users cannot manually configure the compression algorithm, which breaks backward compatibility.

2. ll-killer construction environment and its advantages

In response to the various shortcomings of ll-builder, ll-killer proposes a new building and packaging solution. The core idea is to completely align the build environment with the runtime environment and eliminate unnecessary file copying and caching waste. Key features include:

1. Build environment consistency

• In the project linglong/filesystem maintains application files within the ll-builder only as a base downloader.
• Use ll-killer build The command can repeatedly enter the build environment and the build content will not be reset after exiting.
• The build environment only uses base in binary module completely avoids development dependency issues and ensures consistency between build and runtime environments.

2. The packaging process is efficient

• ll-killer layer build Only 2 to 3 file copies are needed when packaging the command: first run the buildScript script in the project to copy the application files to $PREFIX , reuse mkfs.erofs command creates a layer file and writes the file header.
• Passed mmap+memmove Technology, although subject to mkfs.erofs Although affected by some bugs, the performance of insert operations at the file header is still significantly improved.
• Note: Currently the 3 copies are Due to the bug of mkfs.erofs , ideally 2 times.

Copy path:

1. build构建脚本->$PREFIX(linglong/output/build)
2. app.layer
3. app.layer (Insert layer file header)

3. Disk space and cache management optimization

• Only one copy of the file will be generated during the packaging process, eliminating ~./cache/linglong-builder contains a large amount of cached data that is difficult to clean.
• Support multi-task build, no ostree participates to avoid cache conflicts between multiple projects.

4. Improved speed and flexibility

• compared to ll-builder，ll-killer Build faster. Replication error for
• can be built and packaged on the host, allowing the use of tools on the host, improving overall build efficiency and flexibility.
• You can freely configure various parameters of the layer file, including the compression algorithm.

5. Reliability enhancement

• No custom copy behavior, will not appear ll-builder ( #1039)
• Will not modify the contents of binary files in the build results without authorization (no-strip)

3. Use cases of ll-killer layer subcommand

ll-killer provides multiple subcommands to manage layer files. Common commands and their usage scenarios are introduced below.

Command Overview

$ ll-killer layer --help
独立于玲珑的layer管理器，提供强大的layer文件处理支持，需要安装erofs-utils。

Usage:
  ll-killer layer [command]

Available Commands:
  pack        将文件夹打包为layer。
  build       无需ll-builder, 直接将当前项目构建为layer。
  mount       挂载layer文件。
  umount      卸载layer挂载点。
  dump        输出layer信息。

Flags:
  -h, --help   help for layer

Use "ll-killer layer [command] --help" for more information about a command.
​

• Each subcommand has detailed help documentation, which is recommended.

1. Get ll-killer

In order to smoothly experience the advantages brought by ll-killer, please follow the following steps to download, install and configure:

1. Install necessary dependencies
   Make sure the following dependencies are installed in the system:

   # 用于依赖检查和搜索功能
   sudo apt install apt-file
   ​

2. Download the latest version of ll-killer
   The current latest version is v1.4.3 , the download command is as follows:

   wget https://github.com/System233/ll-killer-go/releases/latest/download/ll-killer-amd64 -O ll-killer
   chmod +x ll-killer
   ​

   The above script downloads amd64 Architecture, ll-killer-go also supports arm64 and loong64 Schema, you can modify the link in amd64 To other versions, or enter the project Releases Page manual download.

   If the link above does not work, you can search online github release download acceleration , then paste the link in to speed up downloading, and then put the file into the corresponding location.

   ⚠Warning: When accelerating downloads from third-party addresses, pay attention to the security of the files. Releases The page provides the SHA1 check code of each file, which must be used after the file is downloaded. sha1sum Verify file integrity.

3. Global installation (optional)
   To facilitate global use, ll-killer can be installed to ~/.local/bin：

   mkdir -p ~/.local/bin
   mv ll-killer ~/.local/bin
   ll-killer -v
   ​

   If it prompts that the command cannot be found or ~/.local/bin is not added to PATH, please execute:

   echo 'export PATH=$HOME/.local/bin:$PATH' >>~/.bashrc
   source ~/.bashrc
   ​

• Hypotheses for subsequent chapters ll-killer Installed globally.

2. The packaging folder is layer

• Function description
  will conform to legal layer structures (see linglong/output/binary directory) are packaged as layers. This command uses mkfs.erofs Package the directory directly, eliminating the need for traditional ll-builder Submit documents to ostree The steps reduce unnecessary file copying and cache generation, protecting disk life.

• Example

  $ tree -L 1 linglong/output/binary
  linglong/output/binary
  ├── curl.install
  ├── entries
  ├── files
  ├── info.json
  └── linglong.yaml
  
  $ ll-killer layer pack linglong/output/binary
  mkfs.erofs 1.8.2
  Build completed.
  ------
  Filesystem UUID: 2e8c228f-d0d3-4eb4-9bfa-db276c6d9f09
  Filesystem total blocks: 10 (of 4096-byte blocks)
  Filesystem total inodes: 30
  Filesystem total metadata blocks: 4
  Filesystem total deduplicated bytes (of source files): 0
  文件已输出至: curl_8.11.1.0_x86_64_binary.layer
  ​

3. Efficiently build Linyaps projects and generate layers

• Function description
  ll-killer layer build Subcommand starts a simulation on the host machine ll-builder The build environment builds the current project as a layer directly on the host machine to avoid unnecessary ostree submission and disk copying.

  • This command is used to supersedes ll-killer in the project ll-builder build+ll-builder export --layer Command combination.
  • If the project is not based on ll-killer , but the build script Do not use any base specific functions , you can also use this command to speed up packaging.
  • If you Hope In build script Using tools from the host , you can use this command to build.
  • 🚀 you Can in command parameters Specify root file system , as specified using Linyaps base The path is as rootfs , if not specified, the root directory of the host will be used.

  This build mode provides the following to simulate an ll-builder environment:

  Environment variables

  LINGLONG_APPID="{APPID}"
  PREFIX="/opt/apps/{APPID}/files"
  TRIPLET="x86_64-linux-gnu|aarch64-linux-gnu|loongarch64-linux-gnu|..." 
  KILLER_PACKER=1 ## killer构建环境标识
  ​

  Table of contents

  • /project : project directory
  • /run/host/rootfs : Same as host
  • / : Same as the specified rootfs.

  Post-processing

  • Added for shortcuts and service units ll-cli run prefix
  • KILLER_PACKER Identifies that it is currently in the killer environment, in the killer environment setup.sh Symbolic link repair will be automatically skipped.
    Can be set before startup KILLER_PACKER=0 Disable this behavior.

• Example

  $ ll-killer layer build
  [准备构建环境]
  Layer元数据:
      名称: curl
      ID: curl
      版本: 8.11.1.0
      模块: binary
      类型: app
      基础: main:org.deepin.base/23.1.0/x86_64
      渠道: main
      命令行: [/opt/apps/curl/files/entrypoint.sh]
      元数据版本: 1.0
      描述: command line tool for transferring data with URL syntax
  curl is a command line tool for transferring data with URL syntax, supporting
  DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3,
  POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP.
  
  curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form
  based upload, proxies, cookies, user+password authentication (Basic, Digest,
  NTLM, Negotiate, kerberos...), file transfer resume, proxy tunneling and a
  busload of other useful tricks.
      架构: x86_64
  [运行构建脚本]
  'test-rel' -> 'test'
  'test-abs' -> '/opt/apps/curl/files/test'
  'file-abs' -> '/opt/apps/curl/files/file'
  'file-rel' -> 'file'
  'noext-abs' -> '/notexist'
  'noext-rel' -> 'notexist'
  'bin-abs' -> '/bin'
  'bin-rel' -> 'bin'
  '/project/applications' -> '/opt/apps/curl/files/share/applications'
  '/project/applications/gimp.desktop' -> '/opt/apps/curl/files/share/applications/gimp.desktop'
  '/project/applications/zutty.desktop' -> '/opt/apps/curl/files/share/applications/zutty.desktop'
  '/project/applications/zutty2.desktop' -> '/opt/apps/curl/files/share/applications/zutty2.desktop'
  '/project/applications/zutty3.desktop' -> '/opt/apps/curl/files/share/applications/zutty3.desktop'
  '/project/applications/zutty4.desktop' -> '/opt/apps/curl/files/share/applications/zutty4.desktop'
  [文件后处理]
  *处理快捷方式和右键菜单
  /opt/apps/curl/files/share/applications/gimp.desktop
  /opt/apps/curl/files/share/applications/zutty.desktop
  /opt/apps/curl/files/share/applications/zutty3.desktop
  /opt/apps/curl/files/share/applications/zutty4.desktop
  /opt/apps/curl/files/share/applications/zutty2.desktop
  [打包输出]
  mkfs.erofs 1.8.2
  Build completed.
  ------
  Filesystem UUID: 7009cb7f-896e-4916-9782-4f17dbfc35a0
  Filesystem total blocks: 6 (of 4096-byte blocks)
  Filesystem total inodes: 25
  Filesystem total metadata blocks: 3
  Filesystem total deduplicated bytes (of source files): 0
  文件已输出至: curl_8.11.1.0_x86_64_binary.layer
  ​

4. Mount and uninstall layer

• Mount layer
  Use this command to mount the layer file to the specified directory for easy viewing and debugging.

  ll-killer layer mount <layer文件名> <挂载目标目录> -- [erofsfuse 选项]
  ​

  Example

  $ ll-killer layer mount curl_8.11.1.0_x86_64_binary.layer curl
  erofsfuse 1.8.2
  <W> erofs: curl_8.11.1.0_x86_64_binary.layer mounted on curl with offset 868
  
  $ ls curl -l
  总计 20
  -rw-r--r-- 1 system system 291  3月11日 17:36 curl.install
  drwxr-xr-x 3 system system  44  3月11日 17:36 entries
  drwxr-xr-x 4 system system 236  3月11日 17:36 files
  -rwxr-xr-x 1 system system 801  3月11日 17:36 info.json
  -rwxr-xr-x 1 system system 763  3月11日 17:36 linglong.yaml
  ​

• Uninstall layer
  When access to the mounted content is no longer needed, use the following command to unmount:

  ll-killer layer umount <挂载目录> -- [fusermount 选项]
  ​

  Example

  $ ll-killer layer umount curl
  $ ls curl -l
  总计 0
  ​

5. Output layer information

• Function description
  This command is used to output detailed information of layer files to help users diagnose and verify layer contents.

• How to use

  ll-killer layer dump <layer文件> -- [dump.erofs 选项]
  ​

  Common parameters:

  • -x : Display file header information;
  • -l : Display Layer information;
  • -e : Display Erofs information;
  • -a : Show all information (enabled by default).

  Example:

  $ ll-killer layer dump curl_8.11.1.0_x86_64_binary.layer 
    Layer文件头:
    文件名: curl_8.11.1.0_x86_64_binary.layer
    文件大小: 25444 字节
    魔数: <<< deepin linglong layer archive >>>
    元数据大小: 824 字节
    数据偏移量: 868
    Layer元数据版本:
        版本: 1
    Layer元数据:
        名称: curl
        ID: curl
        版本: 8.11.1.0
        模块: binary
        类型: app
        基础: main:org.deepin.base/23.1.0/x86_64
        渠道: main
        命令行: [/opt/apps/curl/files/entrypoint.sh]
        元数据版本: 1.0
        描述: command line tool for transferring data with URL syntax
    curl is a command line tool for transferring data with URL syntax, supporting
    DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3,
    POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP.
  
    curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form
    based upload, proxies, cookies, user+password authentication (Basic, Digest,
    NTLM, Negotiate, kerberos...), file transfer resume, proxy tunneling and a
    busload of other useful tricks.
        大小: 32063 字节
        架构: x86_64
    Erofs信息:
    Filesystem magic number:                      0xE0F5E1E2
    Filesystem blocksize:                         4096
    Filesystem blocks:                            6
    Filesystem inode metadata start block:        0
    Filesystem shared xattr metadata start block: 0
    Filesystem root nid:                          36
    Filesystem lz4_max_distance:                  65535
    Filesystem sb_size:                           128
    Filesystem inode count:                       25
    Filesystem created:                           Tue Mar 11 17:36:11 2025
    Filesystem features:                          sb_csum mtime 0padding 
    Filesystem UUID:                              cb6e643e-a762-4310-8d50-61d271f1a51d
  ​

---

4. Summary

ll-killer breaks the limitations of ll-builder in file copying, cache management, multi-task building and environment consistency by reconstructing the build environment, achieving a more efficient and resource-saving build and packaging process.

Whether it is the efficient processing of packaged layers or the convenient operation of building directly on the host, ll-killer has become a powerful tool in the current construction field. Users can get started quickly through simple download and installation steps, and use rich sub-commands to flexibly respond to various scenarios, thereby greatly improving development and release efficiency.

vs. Linyaps ll-builder suggestions ：

ll-killer It has been proven in practice that it is introduced during the construction phase ostree It is 100% negative optimization. It is strongly recommended to delete the build stage. ~/.cache/linglong-builder , only keep base and runtime The download function further reduces the pressure and loss on the disk.