cloudflare / wirefilter

An execution engine for Wireshark-like filters

Wirefilter

This is an execution engine for Wireshark®-like filters.

It contains public APIs for parsing filter syntax, compiling them into an executable IR and, finally, executing filters against provided values.

Example

use wirefilter::{ExecutionContext, Scheme, Type};

fn main() -> Result<(), failure::Error> {
    // Create a map of possible filter fields.
    let scheme = Scheme! {
        http.method: Bytes,
        http.ua: Bytes,
        port: Int,
    };

    // Parse a Wireshark-like expression into an AST.
    let ast = scheme.parse(r#"
        http.method != "POST" &&
        not http.ua matches "(googlebot|facebook)" &&
        port in {80 443}
    "#)?;

    println!("Parsed filter representation: {:?}", ast);

    // Compile the AST into an executable filter.
    let filter = ast.compile();

    // Set runtime field values to test the filter against.
    let mut ctx = ExecutionContext::new(&scheme);

    ctx.set_field_value("http.method", "GET")?;

    ctx.set_field_value(
        "http.ua",
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0",
    )?;

    ctx.set_field_value("port", 443)?;

    // Execute the filter with given runtime values.
    println!("Filter matches: {:?}", filter.execute(&ctx)?); // true

    // Amend one of the runtime values and execute the filter again.
    ctx.set_field_value("port", 8080)?;

    println!("Filter matches: {:?}", filter.execute(&ctx)?); // false

    Ok(())
}

Licensing

Licensed under the MIT license. See the LICENSE file for details.

wirefilter's Issues

Usage issue: Error handling

Hey guys,

Following on from my question yesterday and building on the example you provide I am now having issues getting basic error handling to work. Sorry if this is a newbie question, but I've spend a heap of energy trying to work this out and its just not clicking.

Now when using the failure::Error struct and following the advise I was given here the following line of code:

fn process(rule_data: Json<RuleData>) -> Result<bool, failure::Error> {
    ... 

    let ast =  scheme.parse(&rule_data.rule).map_err(|err|err.to_string())?;
...
}

Is giving me the following error and I cant seam to workout how to deal with this:

error[E0277]: the trait bound `std::string::String: std::error::Error` is not satisfied
  --> src/main.rs:73:75
   |
73 |     let ast =  scheme.parse(&rule_data.rule).map_err(|err|err.to_string())?;
   |                                                                           ^ the trait `std::error::Error` is not implemented for `std::string::String`
   |
   = note: required because of the requirements on the impl of `failure::Fail` for `std::string::String`
   = note: required because of the requirements on the impl of `std::convert::From<std::string::String>` for `failure::error::Error`
   = note: required by `std::convert::From::from`

For the life of me I cant seam to workout how to convert this String or ParseError into a struct that works with the failure::Error struct/package as pre your example.

Sorry if this is a dumb question. New to rust and trying to put this awesome package to use.

Any method to combine multi filter into one to accelerate?

Hello, I'm now writing a package filter demo using wirefilter master branch, with almost 10rules using regex.

I'm curious is there any method to accelerate the match process by combinng multi filter into one to accelerate the process, rather than compare each rule(filter) sequency?

Usage issue: borrowed value does not live long enough

Hey guys,

Rust Newbie here and Am in the process of trying to build a stateless function using your package. However I am getting the below error due to the lifetime usage in the function signature and for the life of me I cant seam to work this one out.

Please see sample code here where I am getting this error:

https://github.com/lukekhamilton/rust-adventures/blob/master/debugging/wirefilter_issue/src/main.rs

Any help greatly appreciated.
L

error[E0597]: `data.rule` does not live long enough
  --> src/main.rs:24:28
   |
24 |     let ast = scheme.parse(&data.rule)?; //  borrowed value does not live long enough??? HELP
   |               -------------^^^^^^^^^^-
   |               |            |
   |               |            borrowed value does not live long enough
   |               argument requires that `data.rule` is borrowed for `'static`
...
36 | }
   | - `data.rule` dropped here while still borrowed

Meta: refine error spans

Currently many errors return spans that last to the end of the input when they could be refined to a narrower region.

Not providing specific examples, but it would be good to walk through the tests and see what can be improved.

Interest in macros to derive the Scheme and ExecutionContext set_field_value logic?

Hello,
Is there any interest in derive macros that:

• Create a Scheme or Vec<(String, Type)> based on a structs fields.
• Auto generates the execution_context.set_field_value logic based on a struct fields.

I have found these useful in my use case, and was curious if there is any interest from the broader community?

Refine .h to use `const *` when possible

Currently Rust side of the FFI needs only immutable references in various functions, but the C headers require mutable pointers in the same positions.

It would be good to align these.

Does wirefilter support BPF target?

In Wireshark filters are compiled to Berkeley Packet Filter which are then mounted to appropriate network interfaces by WinPcap. This avoids copying packets from kernel space to user space.

FFI Example Usage

Would it be possible to have examples of using wirefilter in Go? It's mentioned in the Cloudflare Blog post regarding wirefilter that Cloudflare has embedded the engine into your Go REST API: https://blog.cloudflare.com/how-we-made-firewall-rules/

Issue with Lifetime Parameters in ExecutionContext

Problem:

I'm relatively new to Rust and I've encountered an issue while using a library. It's possible that I might be doing something incorrectly:

The ExecutionContext is using the same lifetime parameter for both the schema and the context parameters.

Currently, my schema has a 'static lifetime, and the context is created for each request. However, the compiler demands that the context also has a 'static lifetime.

pub fn execute<'a>(&self, ctx: &ExecutionContext<'a>) -> bool {
   // `ctx` is a reference that is only valid in the method body
   // lifetime `'a` defined here
   self.filter.execute(ctx).unwrap_or(false)
   // `ctx` escapes the method body here
   // argument requires that `'a` must outlive `'static`
}

Is there something I might be doing incorrectly?

Compilation error in wirefilter-parser

I’m trying to run the tests for this crate, but it doesn’t compile on a current Rust version.

$ cargo test --workspace
(other output...)
error: expected `;`
   --> wirefilter-parser/src/lib.rs:55:18
    |
55  |             $node.children();
    |                  ^
...
125 |         parse_range!(node, int_lit)
    |         --------------------------- in this macro invocation
    |
    = note: this error originates in the macro `proc_macro_call` (in Nightly builds, run with -Z macro-backtrace for more info)

error: expected `;`
   --> wirefilter-parser/src/lib.rs:55:18
    |
55  |             $node.children();
    |                  ^
...
150 |         parse_range!(node, ipv4_lit)
    |         ---------------------------- in this macro invocation
    |
    = note: this error originates in the macro `proc_macro_call` (in Nightly builds, run with -Z macro-backtrace for more info)

error: expected `;`
   --> wirefilter-parser/src/lib.rs:55:18
    |
55  |             $node.children();
    |                  ^
...
155 |         parse_range!(node, ipv6_lit)
    |         ---------------------------- in this macro invocation
    |
    = note: this error originates in the macro `proc_macro_call` (in Nightly builds, run with -Z macro-backtrace for more info)

rustc version: rustc 1.62.1 (e092d0b6b 2022-07-16)
wirefilter version: commit 1fea9c9

Memory Leak Detected on Adding Function to Scheme

Using cargo-valgrind on a project using wirefilter shows a memory leak using Scheme::add_function.

Valgrind output is:

       Error leaked 2.3 kiB in 1 block
        Info at malloc
             at alloc::alloc::alloc (alloc.rs:86)
             at hashbrown::raw::inner::RawTable<T>::new_uninitialized (mod.rs:411)
             at hashbrown::raw::inner::RawTable<T>::fallible_with_capacity (mod.rs:440)
             at hashbrown::raw::inner::RawTable<T>::resize (mod.rs:897)
             at hashbrown::raw::inner::RawTable<T>::reserve_rehash (mod.rs:778)
             at hashbrown::raw::inner::RawTable<T>::reserve (mod.rs:731)
             at hashbrown::raw::inner::RawTable<T>::insert (mod.rs:950)
             at indexmap::map::core::IndexMapCore<K,V>::push (core.rs:214)
             at indexmap::map::core::VacantEntry<K,V>::insert (core.rs:630)
             at wirefilter::scheme::Scheme::add_function (scheme.rs:579)

The exact line that seems to cause the leak is:

wirefilter/engine/src/scheme.rs

Line 579 in 2334ab1

entry.insert(SchemeItem::Function(function.into()));

Is this repository still being updated and maintained?

ffi bindings

Hello!
I try to use this tool in lua via ffi bindings to compiled so-library, but have issues to make it work.
For example, this code works and it is possible to retreive the tool version:

local ffi=require("ffi") 
ffi.cdef[[ 
char* wirefilter_get_version(); 
]] 
local lib=ffi.load("./libwirefilter_ffi.so") 
local version=lib.wirefilter_get_version() 
print(ffi.string(version))

But when i try same approach on funtions that work with RustBox type variables it generates various errors (Segmentation fault, Illegal instruction). Seems that i miss some important part there, so can you give me a hint which variable type in C should be used to correctly work with rust variables (like RustBox)?

Thank you in advance.

Recursive check for nested parameters

Hello!
It is not issue itself, but more like feature request for future: it would be great to perform recursive check of nested parameters.
For example, you have JSON as POST-arguments like {"a":{"b":{"c":1}},"d":2} and it is parsed to request.post.a.b.c = 1 and request.post.d = 2. It would be great to set single rule like request.post ~ "^[0-9]+$" to check all nested elements in request.post variable, instead of making separate checks, adopting rule to request.post.a.b.c ~ "^[0-9]+$" and request.post.d ~"^[0-9]+$".
In context of HTTP protocol, the same might be applicable for GET-arguments, headers, cookies or any parameter that may contain nested structure.

Don't panic if the execution context does not supply all of the fields present in the filter.

Hello,
I ran across this panic while using Wirefilter in my project:
https://github.com/cloudflare/wirefilter/blob/master/engine/src/execution_context.rs#L41

If the field is not set on the ExecutionContext should we just consider it a non matching result? Wanted to discuss this since there are probably cases I am not thinking about (maybe if something is not equal ? ).

Also, if you would like I could take a stab at implementing a fix.

Thank you.

Full syntax support for Cloudflare Firewall Rules language

Hi!

I've been looking into writing unit tests for firewall rules using wirefilter. However, as far as I can see, wirefilter doesn't support the full syntax of Cloudflare Firewall Rules language. For instance, as far as can I can see it's not possible to express complex types, e.g. arrays, as well as transformation functions.

Do you know if wirefilter is planning to support the full cloudflare syntax, or is it going to support the current set of expressions only?