Update LibAFL by tokatoka · Pull Request #1902 · google/fuzzbench

tokatoka · 2023-09-22T17:37:09Z

and the rust compiler it uses

CC @andreafioraldi @addisoncrump

…date_libafl

addisoncrump · 2023-09-22T18:45:09Z

So to fix that last one you can use a fuzzbench-specific profile like I do here:

tokatoka · 2023-09-22T18:46:55Z

OK 👍

addisoncrump · 2023-09-23T13:38:26Z

#13 62.01 error[E0512]: cannot transmute between types of different sizes, or dependently-sized types
#13 62.01   --> /libafl/libafl/src/bolts/anymap.rs:59:5
#13 62.01    |
#13 62.01 59 |     assert_eq_size!(TypeId, u64);
#13 62.01    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#13 62.01    |
#13 62.01    = note: source type: `TypeId` (128 bits)
#13 62.01    = note: target type: `u64` (64 bits)
#13 62.01    = note: this error originates in the macro `assert_eq_size` (in Nightly builds, run with -Z macro-backtrace for more info)
#13 62.01

We fixed this a while ago; old version of LibAFL being used?

tokatoka · 2023-09-23T13:49:13Z

I changed fuzzers/libafl. but for some reason the fuzzer being built is libafl_forkserver (Why?)

addisoncrump · 2023-09-23T14:05:48Z

Yeah that's weird. @jonathanmetzman @alan32liu there seems to be a CI bug here.

In the meantime, can we just update the commit for the other libafls?

tokatoka · 2023-09-23T14:09:02Z

Yes
I deleted libafl_forkserver because it's just the same as aflrustrust
and updated the commit

tokatoka · 2023-09-23T21:46:17Z

now good 😊

DonggeLiu · 2023-09-25T01:38:06Z

now good 😊

Not sure what has happened with the CI tests, but do they use the correct fuzzer now?
I am happy to merge it if that is the case.

tokatoka · 2023-09-25T12:07:56Z

Not sure what has happened with the CI tests, but do they use the correct fuzzer now?

No
I edited ['libafl'], but the CI runs all the tests for ['libafl', 'aflrustrust', 'libafl_forkserver'].
CI looks fine because in the end I deleted 'libafl_forkserver' and patched 'aflfurst'

addisoncrump · 2023-09-25T12:08:19Z

now good 😊

Not sure what has happened with the CI tests, but do they use the correct fuzzer now? I am happy to merge it if that is the case.

The CI issue is likely still present, but @tokatoka deleted the older fuzzer in favour of the new.

DonggeLiu · 2023-09-25T12:18:40Z

I reckon that is because aflrustrust and libafl_forkserver depend on libafl.

This is an intended behaviour in FuzzBench, because it wants to make sure that changing the base fuzzer will not silently break other fuzzers that depend on it.

tokatoka · 2023-09-25T12:21:10Z

I see 👌
You can merge this now.

DonggeLiu · 2023-09-25T12:28:08Z

I see 👌 You can merge this now.

Sure! But let's double-check with @andreafioraldi first :)

DonggeLiu · 2023-09-25T12:32:46Z

BTW, given this affects the master branch, would you think it makes sense to run an experiment to ensure the changes work as we expected before merging?

I guess you already know how to make a trivial modification to service/gcbrun_experiment.py and write your experiment request in this format : )
Thanks!

tokatoka · 2023-09-25T12:45:39Z

BTW, given this affects the master branch, would you think it makes sense to run an experiment to ensure the changes work as we expected before merging?

OK i'll push necessary changes now

DonggeLiu · 2023-09-25T12:46:43Z

OK i'll push necessary changes now

Thanks!

…date_libafl

tokatoka · 2023-09-25T13:16:12Z

I added the old fuzzer using the old commit (slightly changed because it doesn't one of its dependency crate)

The command would be

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-09-25-libafl --fuzzers libafl libafl_27042023

DonggeLiu · 2023-09-27T11:11:50Z

now you can merge this 🙏

Would it make sense to run another experiment, just to triple-check?

tokatoka · 2023-09-27T11:13:03Z

okay 😂

This is the command;

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-09-27-libafl --fuzzers libafl libafl_27042023 aflrustrust

DonggeLiu · 2023-09-27T11:13:49Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-09-27-libafl --fuzzers libafl libafl_27042023 aflrustrust

tokatoka · 2023-09-27T14:01:20Z

Now the experiment has run again. but the result has not improved.

and in the build log benchmark-libpcap_fuzz_both-fuzzer-libafl.txt

Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate": 
Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate": #7 [4/7] RUN git clone https://github.com/AFLplusplus/LibAFL /libafl
Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate": #7 sha256:176f27286eece531de3dbf978cee7faecdf4771dd0fce583799967f009b4217a
Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate": #7 pulling sha256:dcd3c5754ac77fd04a705522927db13aa3e5c571d655e7d0a2162db6db0eef18
Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate": #7 pulling sha256:d4d55b1b8b30662f75a7938283a0c970af9b6b309b55508584a92b95a7ac929e
Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate": #7 pulling sha256:d1ef8f825a26cbed27232333e2c285dba737bf5fa2cdec25c4db969b9a3ba462
Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate": #7 pulling sha256:d1ef8f825a26cbed27232333e2c285dba737bf5fa2cdec25c4db969b9a3ba462 0.3s done
Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate": #7 pulling sha256:dcd3c5754ac77fd04a705522927db13aa3e5c571d655e7d0a2162db6db0eef18 2.6s done
Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate": #7 pulling sha256:d4d55b1b8b30662f75a7938283a0c970af9b6b309b55508584a92b95a7ac929e 5.0s done
Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate": #7 CACHED
Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate": 
Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate": #8 [5/7] RUN cd /libafl && git checkout 324db072a2f0a25f4e891d8a008a2feff240fb79 || true
Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate": #8 sha256:c5f24e432e51a041c7f098db02b62686e3b36711d2f93ba8a653991fa977f63d
Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate": #8 0.294 fatal: reference is not a tree: 324db072a2f0a25f4e891d8a008a2feff240fb79
Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate": #8 DONE 9.0s
Step #2 - "libafl-libpcap_fuzz_both-builder-intermediate":

324db072a2f0a25f4e891d8a008a2feff240fb79 is the commit the includes the commit.
but it says it can't checkout to the latest libafl because it is using "CACHED" old clone of libafl. and this guy doesn't contain my latest commit
Is there a way to tell the docker to stop using cache?

DonggeLiu · 2023-09-28T04:03:04Z

324db072a2f0a25f4e891d8a008a2feff240fb79 is the commit the includes the commit.
but it says it can't checkout to the latest libafl because it is using "CACHED" old clone of libafl. and this guy doesn't contain my latest commit

How about adding a git pull before git checkout?

tokatoka · 2023-09-28T09:19:44Z

ok

DonggeLiu · 2023-09-28T10:24:58Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-09-28-libafl --fuzzers libafl libafl_27042023 aflrustrust

DonggeLiu · 2023-09-28T10:30:27Z

Experiment 2023-09-28-libafl data and results will be available later at:
The experiment data.
The experiment report.

jonathanmetzman · 2023-09-29T20:04:33Z

ok

I think sometimes you need to modify the file to skip this caching. I'm not sure git pull will help here.

tokatoka · 2023-10-02T12:15:34Z

this pr is ready (for the 3rd time)

DonggeLiu · 2023-10-04T02:47:29Z

Experiment 2023-09-28-libafl data and results will be available later at: The experiment data. The experiment report.

This report shows [libafl_27042023](https://github.com/google/fuzzbench/blob/master/fuzzers/libafl_27042023) performs better than libafl.
Is that because git pull does not work?
Shall we re-run the exp after you update the file (which should avoid using cache)?

tokatoka · 2023-10-04T10:51:50Z

Yes indeed 2704 was better. especially with libjpeg
We identified the cause and fixed some bugs in the latest commit of this PR. git pull worked for us.

Shall we re-run the exp after you update the file (which should avoid using cache)?

Yes please 🙏

DonggeLiu · 2023-10-05T00:34:56Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-10-05-libafl --fuzzers libafl libafl_27042023 aflrustrust

DonggeLiu · 2023-10-05T00:55:27Z

Experiment 2023-10-05-libafl data and results will be available later at:
The experiment data.
The experiment report.

tokatoka · 2023-10-06T11:04:58Z

this one looks good now we can merge it 🙏

DonggeLiu · 2023-10-08T22:40:30Z

this one looks good now we can merge it 🙏

Sure, could you please fix the format error in CI?
Here is the log:

source .venv/bin/activate && python3 presubmit.py
--- /home/runner/work/fuzzbench/fuzzbench/fuzzers/libafl/fuzzer.py	(original)
+++ /home/runner/work/fuzzbench/fuzzbench/fuzzers/libafl/fuzzer.py	(reformatted)
@@ -66,6 +66,6 @@
         command += (['-x', dictionary_path])
     command += (['-o', output_corpus, '-i', input_corpus])
     fuzzer_env = os.environ.copy()
-    fuzzer_env['LD_PRELOAD']='/usr/lib/x[8](https://github.com/google/fuzzbench/actions/runs/6379929561/job/17313360212?pr=1902#step:8:9)6_64-linux-gnu/libjemalloc.so.2'
+    fuzzer_env['LD_PRELOAD'] = '/usr/lib/x86_64-linux-gnu/libjemalloc.so.2'
     print(command)
     subprocess.check_call(command, cwd=os.environ['OUT'], env=fuzzer_env)


Code is not formatted correctly, please run 'make format'
ERROR: format failed, see errors above.
Failed checks: format
Failed.
make: *** [Makefile:60: presubmit] Error 1
Error: Process completed with exit code 2.

tokatoka · 2023-10-12T18:23:14Z

Done 🙏

DonggeLiu · 2023-10-12T23:59:59Z

/gcbrun skip

DonggeLiu · 2023-10-13T00:00:16Z

Done 🙏

Thanks!

@andreafioraldi

and the rust compiler it uses CC @andreafioraldi @addisoncrump

tokatoka added 2 commits September 22, 2023 19:35

update

32c3417

Merge branch 'update_libafl' of github.com:tokatoka/fuzzbench into up…

c8e81e9

…date_libafl

addisoncrump mentioned this pull request Sep 22, 2023

libafl_libfuzzer optimization experiment #1903

Closed

tokatoka added 5 commits September 22, 2023 23:52

Update builder.Dockerfile

27a796a

Update builder.Dockerfile

9c40224

last fix

e15c33b

rust version?

4dc7f9d

FMT

1118508

Delete libafl_forkserver, update commit for aflrustrust

04d430c

andreafioraldi approved these changes Sep 25, 2023

View reviewed changes

tokatoka added 3 commits September 25, 2023 15:11

Add Old Libfuzzer

712a4fe

Merge branch 'update_libafl' of github.com:tokatoka/fuzzbench into up…

d1b7d4b

…date_libafl

A DUMMY COMMENT

63916ac

DonggeLiu mentioned this pull request Sep 28, 2023

Using outdated cached fuzzer repo. #1905

Open

git pull

2352d4a

tokatoka marked this pull request as draft September 29, 2023 19:56

use JEMALLOC and update

3c53c68

tokatoka marked this pull request as ready for review October 2, 2023 12:15

FMT

3d64992

DonggeLiu enabled auto-merge (squash) October 13, 2023 00:00

DonggeLiu disabled auto-merge October 13, 2023 01:25

DonggeLiu merged commit ba22647 into google:master Oct 13, 2023

addisoncrump mentioned this pull request Mar 15, 2024

delete fuzzbench profile AFLplusplus/LibAFL#1941

Closed

SPINO-11 pushed a commit to SPINO-11/green_fuzzing_evaluation that referenced this pull request Aug 9, 2025

Update LibAFL (google#1902)

1a98c5f

and the rust compiler it uses CC @andreafioraldi @addisoncrump

Conversation

tokatoka commented Sep 22, 2023

Uh oh!

addisoncrump commented Sep 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tokatoka commented Sep 22, 2023

Uh oh!

addisoncrump commented Sep 23, 2023

Uh oh!

tokatoka commented Sep 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

addisoncrump commented Sep 23, 2023

Uh oh!

tokatoka commented Sep 23, 2023

Uh oh!

tokatoka commented Sep 23, 2023

Uh oh!

DonggeLiu commented Sep 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tokatoka commented Sep 25, 2023

Uh oh!

addisoncrump commented Sep 25, 2023

Uh oh!

DonggeLiu commented Sep 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tokatoka commented Sep 25, 2023

Uh oh!

DonggeLiu commented Sep 25, 2023

Uh oh!

DonggeLiu commented Sep 25, 2023

Uh oh!

tokatoka commented Sep 25, 2023

Uh oh!

DonggeLiu commented Sep 25, 2023

Uh oh!

tokatoka commented Sep 25, 2023

Uh oh!

DonggeLiu commented Sep 27, 2023

Uh oh!

tokatoka commented Sep 27, 2023

Uh oh!

DonggeLiu commented Sep 27, 2023

Uh oh!

tokatoka commented Sep 27, 2023

Uh oh!

DonggeLiu commented Sep 28, 2023

Uh oh!

tokatoka commented Sep 28, 2023

Uh oh!

DonggeLiu commented Sep 28, 2023

Uh oh!

DonggeLiu commented Sep 28, 2023

Uh oh!

jonathanmetzman commented Sep 29, 2023

Uh oh!

tokatoka commented Oct 2, 2023

Uh oh!

DonggeLiu commented Oct 4, 2023

Uh oh!

tokatoka commented Oct 4, 2023

Uh oh!

DonggeLiu commented Oct 5, 2023

Uh oh!

DonggeLiu commented Oct 5, 2023

Uh oh!

tokatoka commented Oct 6, 2023

Uh oh!

DonggeLiu commented Oct 8, 2023

Uh oh!

tokatoka commented Oct 12, 2023

Uh oh!

DonggeLiu commented Oct 12, 2023

Uh oh!

DonggeLiu commented Oct 13, 2023

Uh oh!

Reviewers

addisoncrump commented Sep 22, 2023 •

edited

Loading

tokatoka commented Sep 23, 2023 •

edited

Loading

DonggeLiu commented Sep 25, 2023 •

edited

Loading

DonggeLiu commented Sep 25, 2023 •

edited

Loading