-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Support integration with luzer #13929
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,185 @@ | ||
| --- | ||
| layout: default | ||
| title: Integrating a Lua project | ||
| parent: Setting up a new project | ||
| grand_parent: Getting started | ||
| nav_order: 4 | ||
| permalink: /getting-started/new-project-guide/lua-lang/ | ||
| --- | ||
|
|
||
| # Integrating a Lua project | ||
| {: .no_toc} | ||
|
|
||
| - TOC | ||
| {:toc} | ||
| --- | ||
|
|
||
| The process of integrating a project written in Lua with OSS-Fuzz | ||
| is similar to the general [Setting up a new project]({{ site.baseurl | ||
| }}/getting-started/new-project-guide/) process. The key specifics of | ||
| integrating a Lua project are outlined below. | ||
|
|
||
| ## luzer | ||
|
|
||
| Lua fuzzing in OSS-Fuzz is powered by | ||
| [luzer](https://github.com/ligurio/luzer). As luzer operates | ||
| directly on the Lua source code level, it can be applied to any | ||
| project written in a language that can be transpiled into Lua, | ||
| such as [MoonScript](https://moonscript.org/), | ||
| [TypeScriptToLua](https://typescripttolua.github.io/), | ||
| [Fennel](https://fennel-lang.org/), and [Urn](https://urn-lang.com/). | ||
| Also, it supports fuzzing C/C++ extensions written for Lua. When | ||
| fuzzing native code, luzer can be used in combination with | ||
| Address Sanitizer or Undefined Behavior Sanitizer to catch extra bugs. | ||
|
|
||
| ## Project files | ||
|
|
||
| ### Example project | ||
|
|
||
| We recommend viewing | ||
| [lua-example](https://github.com/google/oss-fuzz/tree/master/projects/lua-example) | ||
| as an example of a simple Lua fuzzing project. This example also | ||
| demonstrates how to use luzer's Fuzzed Data Provider. | ||
|
|
||
| ### project.yaml | ||
|
|
||
| The `language` attribute must be specified as follows: | ||
|
|
||
| ```yaml | ||
| language: c | ||
| ``` | ||
|
|
||
| The only supported fuzzing engine is libFuzzer (`libfuzzer`). | ||
|
|
||
| ```yaml | ||
| fuzzing_engines: | ||
| - libfuzzer | ||
| sanitizers: | ||
| - none | ||
| ``` | ||
|
|
||
| There is nothing special for sanitizer support in OSS-Fuzz | ||
| infrastructure. luzer builds its own DSO with libFuzzer and | ||
| sanitizer and `compile_lua_fuzzer` (also managed by project) sets | ||
| it to `LD_PRELOAD` if required. | ||
|
|
||
| ### Dockerfile | ||
|
|
||
| The Dockerfile should start by `FROM gcr.io/oss-fuzz-base/base-builder`. | ||
|
|
||
| The OSS-Fuzz base Docker images come without any pre-installed | ||
| components required for Lua fuzzing. Apart from that, you should | ||
| usually need to build or install a Lua runtime, luzer module, | ||
| clone the project, set a `WORKDIR`, and copy any necessary files, | ||
| or install any project-specific dependencies here as you normally would. | ||
|
|
||
| ### Fuzzers | ||
|
|
||
| In the simplest case, every fuzzer consists of a single Lua file that defines | ||
| a function `TestOneInput` and executes a function named `luzer.Fuzz()`. | ||
| An example fuzz target could thus be a file `fuzz_basic.lua` with contents: | ||
|
|
||
| ```lua | ||
| local parser = require("src.luacheck.parser") | ||
| local decoder = require("luacheck.decoder") | ||
| local luzer = require("luzer") | ||
|
|
||
| local function TestOneInput(buf) | ||
| parser.parse(decoder.decode(buf)) | ||
| end | ||
|
|
||
| local args = { | ||
| print_final_stats = 1, | ||
| } | ||
| luzer.Fuzz(TestOneInput, nil, args) | ||
| ``` | ||
|
|
||
| ### compile_lua_fuzzer | ||
|
|
||
| Unlike projects for other languages, the base image does not | ||
| include a script that generates a wrapper script that can be used | ||
| as a drop-in replacement for libFuzzer. | ||
|
|
||
| Therefore, you need to add such a script yourself. This script | ||
| sets a relative path to Lua runtime that will be used for running | ||
| tests and the necessary environment variables (for example, `LUA_PATH`, | ||
| `LUA_CPATH` and `LD_PRELOAD`) and specifies the path directly to | ||
| the `.lua` file containing the test implementation. The script | ||
| `compile_lua_fuzzer` must accept the same command line flags as | ||
| libFuzzer-based tests. | ||
|
|
||
| Note, the resulting wrapper scripts must contain the word "luarocks" | ||
| to pass checks by `bad_build_check` in continuous integration. | ||
|
|
||
| Then, you can use the script `compile_lua_fuzzer` to build the fuzzers. | ||
| A usage example from the `lua-example` project is | ||
|
|
||
| ```shell | ||
| compile_lua_fuzzer lua fuzz_basic.lua | ||
| ``` | ||
|
|
||
| Arguments are: | ||
|
|
||
| * a relative path to a Lua runtime name | ||
| * a relative path to the fuzzing test inside the OSS Fuzz project directory | ||
|
|
||
| The `lua-example` projects includes an | ||
| [example](https://github.com/google/oss-fuzz/blob/master/projects/lua-example/compile_lua_fuzzer) | ||
| of such script. | ||
|
|
||
| ### build.sh | ||
|
|
||
| The script is executed within the image built from your [Dockerfile](#Dockerfile). | ||
|
|
||
| In general, this script should do the following: | ||
|
|
||
| - Set up or build a Lua runtime. | ||
| - Set up or build required dependencies for your tests. | ||
| - Generate wrapper scripts for your tests using [compile_lua_fuzzer](#compile_lua_fuzzer). | ||
|
|
||
| Resulting binaries, tests and their wrapper scripts, and a | ||
| directory with Luarocks dependencies should be placed in `$OUT`. | ||
|
|
||
| Beware, when installing the luzer module, you need to set the | ||
| environment variable `OSS_FUZZ` to non-empty value, otherwise the | ||
| build may fail. | ||
|
|
||
| The [lua-example](https://github.com/google/oss-fuzz/blob/master/projects/lua-example/build.sh) | ||
| project contains an example of a `build.sh` for a Lua projects. | ||
|
|
||
| ## FuzzedDataProvider | ||
|
|
||
| luzer provides a Fuzzed Data Provider that is helpful for splitting | ||
| a fuzz input into multiple parts of various Lua types. Its | ||
| functionality is similar to | ||
| [Fuzzed Data Provider](https://github.com/google/fuzzing/blob/master/docs/split-inputs.md#fuzzed-data-provider) | ||
| available in LLVM. Learn about methods, provided by FDP in luzer, | ||
| in [documentation](https://github.com/ligurio/luzer/blob/master/docs/api.md#structure-aware-fuzzing). | ||
|
|
||
| A fuzz target using the `FuzzedDataProvider` would look as follows: | ||
|
|
||
| ```lua | ||
| local luzer = require("luzer") | ||
|
|
||
| local function TestOneInput(buf) | ||
| local fdp = luzer.FuzzedDataProvider(buf) | ||
| local str = fdp:consume_string(4) | ||
|
|
||
| local b = {} | ||
| str:gsub(".", function(c) table.insert(b, c) end) | ||
| local count = 0 | ||
| if b[1] == "o" then count = count + 1 end | ||
| if b[2] == "o" then count = count + 1 end | ||
| if b[3] == "p" then count = count + 1 end | ||
| if b[4] == "s" then count = count + 1 end | ||
|
|
||
| if count == 4 then assert(nil) end | ||
| end | ||
|
|
||
| local args = { | ||
| only_ascii = 1, | ||
| print_pcs = 1, | ||
| } | ||
|
|
||
| luzer.Fuzz(TestOneInput, nil, args) | ||
| ``` | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| # Copyright 2023-2026 Google LLC | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| # | ||
| ################################################################################ | ||
|
|
||
| FROM gcr.io/oss-fuzz-base/base-builder | ||
|
|
||
| RUN apt-get update && apt-get install -y build-essential make | ||
|
|
||
| COPY build.sh $SRC/ | ||
| COPY compile_lua_fuzzer $SRC/ | ||
|
|
||
| # For real projects, you would clone your repo in the next step. | ||
| RUN mkdir -p $SRC/example | ||
|
|
||
| # Ideally, you have already configured fuzz tests in your repo so | ||
| # that they run (in luzer regression mode) as part of unit testing. | ||
| # Keeping the fuzz tests in sync with the source code ensures that | ||
| # they are adjusted continue to work after code changes. Here, | ||
| # we copy them into the example project directory. | ||
| COPY example_basic.lua $SRC/example/fuzz_basic.lua | ||
|
|
||
| WORKDIR $SRC/example |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| #!/bin/bash -eu | ||
|
|
||
| # Copyright 2026 Google LLC | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| # | ||
| ################################################################################ | ||
|
|
||
| # Finish execution if libFuzzer is not used, because luzer | ||
| # is libFuzzer-based. | ||
| if [[ "$FUZZING_ENGINE" != libfuzzer ]]; then | ||
| return | ||
| fi | ||
|
|
||
| # Install dependencies: | ||
| # 1. The libraries required for building luzer module. | ||
| # 2. PUC Rio Lua executable, required for running tests itself. | ||
| # 3. The Lua package manager. | ||
| apt install -y cmake liblua5.1-0 liblua5.1-0-dev lua5.1 luarocks | ||
|
|
||
| export OSS_FUZZ=1 | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @jonathanmetzman The OSS Fuzz environment has some differences 1 in comparison to usual Linux environment (for example, a name of sanitizers libraries and Footnotes
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Feel free to add one to base-images/base
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I wonder if it would make sense to define |
||
| # Install Lua libraries. | ||
| luarocks install --lua-version 5.1 --server=https://luarocks.org/dev --tree=lua_modules luzer | ||
|
|
||
| LUA_RUNTIME_NAME=lua | ||
|
|
||
| # Build fuzzers. | ||
| $SRC/compile_lua_fuzzer $LUA_RUNTIME_NAME fuzz_basic.lua | ||
|
|
||
| cp fuzz_basic.lua "$OUT/" | ||
| cp /usr/bin/lua5.1 "$OUT/$LUA_RUNTIME_NAME" | ||
| cp -R lua_modules "$OUT/" | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| #!/bin/bash -eu | ||
| # Copyright 2026 Google LLC | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| # | ||
| ################################################################################ | ||
|
|
||
| # The Lua runtime name. | ||
| lua_runtime=$1 | ||
| # Path to the fuzz target source file relative to the project's root. | ||
| fuzz_target=$2 | ||
|
|
||
| fuzzer_basename=$(basename -s .lua "$fuzz_target") | ||
|
|
||
| # Create an execution wrapper that executes luzer with the correct | ||
| # arguments. | ||
| echo "#!/bin/bash | ||
|
|
||
| # LLVMFuzzerTestOneInput so that the wrapper script is recognized | ||
| # as a fuzz target for 'check_build'. | ||
| project_dir=\$(dirname \"\$0\") | ||
| eval \$(luarocks --tree lua_modules path) | ||
| LD_PRELOAD=\$project_dir/lua_modules/lib/lua/5.1/libfuzzer_with_asan.so \ | ||
| ASAN_OPTIONS=\$ASAN_OPTIONS:symbolize=1:external_symbolizer_path=\$project_dir/llvm-symbolizer:detect_leaks=0 \ | ||
| \$project_dir/$lua_runtime \$project_dir/$fuzz_target \$@" > "$OUT/$fuzzer_basename" | ||
|
|
||
| chmod +x "$OUT/$fuzzer_basename" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| -- Copyright 2026 Google LLC | ||
|
|
||
| -- Licensed under the Apache License, Version 2.0 (the "License"); | ||
| -- you may not use this file except in compliance with the | ||
| -- License. | ||
| -- You may obtain a copy of the License at | ||
| -- | ||
| -- http://www.apache.org/licenses/LICENSE-2.0 | ||
| -- | ||
| -- Unless required by applicable law or agreed to in writing, | ||
| -- software distributed under the License is distributed on an | ||
| -- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, | ||
| -- either express or implied. | ||
| -- See the License for the specific language governing permissions | ||
| -- and limitations under the License. | ||
|
|
||
| local luzer = require("luzer") | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Needs a license header
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed: --- a/projects/lua-example/example_basic.lua
+++ b/projects/lua-example/example_basic.lua
@@ -1,3 +1,19 @@
+-- Copyright 2023-2025 Google LLC
+
+-- Licensed under the Apache License, Version 2.0 (the "License");
+-- you may not use this file except in compliance with the
+-- License.
+-- You may obtain a copy of the License at
+
+-- http://www.apache.org/licenses/LICENSE-2.0
+
+-- Unless required by applicable law or agreed to in writing,
+-- software distributed under the License is distributed on an
+-- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
+-- either express or implied.
+-- See the License for the specific language governing permissions
+-- and limitations under the License.
+
local luzer = require("luzer")
local function TestOneInput(buf) |
||
|
|
||
| local function TestOneInput(buf) | ||
| local fdp = luzer.FuzzedDataProvider(buf) | ||
| local str = fdp:consume_string(4) | ||
|
|
||
| local b = {} | ||
| str:gsub(".", function(c) table.insert(b, c) end) | ||
| local count = 0 | ||
| if b[1] == "o" then count = count + 1 end | ||
| if b[2] == "o" then count = count + 1 end | ||
| if b[3] == "p" then count = count + 1 end | ||
| if b[4] == "s" then count = count + 1 end | ||
|
|
||
| if count == 4 then assert(nil) end | ||
| end | ||
|
|
||
| local args = { | ||
| only_ascii = 1, | ||
| print_pcs = 1, | ||
| } | ||
|
|
||
| luzer.Fuzz(TestOneInput, nil, args) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Errr...I hope this combo works. It's uncommon.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've removed it, anyway nothing special is required from infra regarding sanitizers. luzer builds its own DSO libFuzzer+sanitizer and
compile_lua_fuzzer(also managed by project) sets it toLD_PRELOADif required.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BTW,
noneis used in a chapter for JavaScript projects, seeoss-fuzz/docs/getting-started/new-project-guide/javascript_lang.md
Lines 61 to 62 in 66f567e
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It was a real question. I'm just not sure :-)