new AFL++ mutation engine tests by vanhauser-thc · Pull Request #1866 · google/fuzzbench

vanhauser-thc · 2023-06-21T07:56:45Z

No description provided.

vanhauser-thc · 2023-06-21T09:22:02Z

@alan32liu this can be merged, thank you!

DonggeLiu · 2023-06-21T11:43:58Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-06-23-aflpp --fuzzers aflplusplus aflplusplusplus aflplusplus_orig aflplusplus_orignewtext aflplusplus_text aflplusplus_exploit

DonggeLiu · 2023-06-21T11:45:38Z

Ah, I have not fixed the issue yet.
Could we launch experiments here and merge?
As you know, we need a dummy change to do that :)

vanhauser-thc · 2023-06-21T12:07:05Z

Ah, I have not fixed the issue yet. Could we launch experiments here and merge? As you know, we need a dummy change to do that :)

done :)

DonggeLiu · 2023-06-21T12:08:08Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-06-23-aflpp --fuzzers aflplusplus aflplusplusplus aflplusplus_orig aflplusplus_orignewtext aflplusplus_text aflplusplus_exploit

DonggeLiu · 2023-06-21T12:28:06Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-06-23-aflpp-bug --fuzzers aflplusplus aflplusplusplus aflplusplus_orig aflplusplus_orignewtext aflplusplus_text aflplusplus_exploit --benchmarks bloaty_fuzz_target_52948c harfbuzz_hb-shape-fuzzer_17863b libxml2_xml_e85b9b mbedtls_fuzz_dtlsclient_7c6b0e php_php-fuzz-parser_0dbedb

vanhauser-thc · 2023-06-23T10:08:03Z

@alan32liu
can you please execute for me:

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-06-24-aflpp --fuzzers aflplusplus aflplusplus_switch5 aflplusplus_switch30 aflplusplus_explore

thanks!

vanhauser-thc · 2023-06-23T10:23:00Z

@alan32liu
and this :-)

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-06-25-aflpp --fuzzers aflplusplus_explore aflplusplus_exploit aflplusplus_explore_txt aflplusplus_exploit_txt

thanks a lot!

vanhauser-thc · 2023-06-24T11:42:39Z

and this one, this should be the final one hopefully for the new mutation engine. :-) thanks a lot!

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-06-26-aflpp --fuzzers aflplusplus_new_bin aflplusplus_new_txt aflplusplus_orignewtext

DonggeLiu · 2023-06-25T23:56:43Z

Ah I just saw this.
I will execute them in order now.

DonggeLiu · 2023-06-25T23:56:47Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-06-24-aflpp --fuzzers aflplusplus aflplusplus_switch5 aflplusplus_switch30 aflplusplus_explore

DonggeLiu · 2023-06-26T00:00:54Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-06-25-aflpp --fuzzers aflplusplus_explore aflplusplus_exploit aflplusplus_explore_txt aflplusplus_exploit_txt

DonggeLiu · 2023-06-26T00:04:11Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-06-26-aflpp --fuzzers aflplusplus_new_bin aflplusplus_new_txt aflplusplus_orignewtext

vanhauser-thc · 2023-06-28T15:40:41Z

@alan32liu thank you that helped a lot!
Now I only need a bug benchmark run with:

aflplusplus_exploit aflplusplus_exploit_txt aflplusplus_explore aflplusplus_explore_txt aflplusplus_new_bin aflplusplus_new_txt aflplusplus_orignewtext aflplusplus_orig

and then I am done with the new mutation engine :-)

DonggeLiu · 2023-06-29T00:32:41Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-06-26-aflpp --fuzzers aflplusplus_exploit aflplusplus_exploit_txt aflplusplus_explore aflplusplus_explore_txt aflplusplus_new_bin aflplusplus_new_txt aflplusplus_orignewtext aflplusplus_orig

vanhauser-thc · 2023-06-29T06:49:01Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-06-26-aflpp --fuzzers aflplusplus_exploit aflplusplus_exploit_txt aflplusplus_explore aflplusplus_explore_txt aflplusplus_new_bin aflplusplus_new_txt aflplusplus_orignewtext aflplusplus_orig

thanks :) that is a normal run, not a bug benchmark though? and doesnt that experiment name already exist?

DonggeLiu · 2023-06-29T23:47:55Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-06-30-aflpp-bug --fuzzers aflplusplus_exploit aflplusplus_exploit_txt aflplusplus_explore aflplusplus_explore_txt aflplusplus_new_bin aflplusplus_new_txt aflplusplus_orignewtext aflplusplus_orig

DonggeLiu · 2023-06-29T23:49:44Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-06-26-aflpp --fuzzers aflplusplus_exploit aflplusplus_exploit_txt aflplusplus_explore aflplusplus_explore_txt aflplusplus_new_bin aflplusplus_new_txt aflplusplus_orignewtext aflplusplus_orig

thanks :) that is a normal run, not a bug benchmark though? and doesnt that experiment name already exist?

Ah, silly me, I forgot to change the experiment name.
When launching experiments in this way, their types (cov, bug) are determined by the benchmarks. I don't think any extra flag is required to specify its type.

vanhauser-thc · 2023-06-30T04:54:51Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-06-26-aflpp --fuzzers aflplusplus_exploit aflplusplus_exploit_txt aflplusplus_explore aflplusplus_explore_txt aflplusplus_new_bin aflplusplus_new_txt aflplusplus_orignewtext aflplusplus_orig

thanks :) that is a normal run, not a bug benchmark though? and doesnt that experiment name already exist?

Ah, silly me, I forgot to change the experiment name. When launching experiments in this way, their types (cov, bug) are determined by the benchmarks. I don't think any extra flag is required to specify its type.

Changing the name helped to start the experiment however it is not doing a bug benchmark…

DonggeLiu · 2023-06-30T04:57:18Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-06-30-aflpp-bug-1 --fuzzers aflplusplus_exploit aflplusplus_exploit_txt aflplusplus_explore aflplusplus_explore_txt aflplusplus_new_bin aflplusplus_new_txt aflplusplus_orignewtext aflplusplus_orig --benchmarks bloaty_fuzz_target_52948c harfbuzz_hb-shape-fuzzer_17863b libxml2_xml_e85b9b mbedtls_fuzz_dtlsclient_7c6b0e php_php-fuzz-parser_0dbedb

DonggeLiu · 2023-06-30T05:08:52Z

Changing the name helped to start the experiment however it is not doing a bug benchmark…

The new one should be correct. I also deleted the VM instances of the previous one.
Sorry that I was not thinking clearly when launch the previous one.

vanhauser-thc · 2023-06-30T08:07:58Z

thank you!! :)

DonggeLiu · 2023-07-20T04:10:05Z

wow I can really start the experiment myself ... is that ok?

I would not mind casual ones : )

vanhauser-thc · 2023-07-21T08:07:05Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-07-21-aflpp --fuzzers aflplusplus aflplusplus_func aflplusplus_loop

vanhauser-thc · 2023-07-21T16:46:59Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-07-22-aflpp --fuzzers aflplusplus aflplusplus_func aflplusplus_loop

vanhauser-thc · 2023-07-27T12:34:03Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-07-27-aflpp-bug --fuzzers aflplusplus aflplusplus_func_exploit aflplusplus_loop_exploit aflplusplus_func aflplusplus_loop aflplusplus_loopfunc_exploit --benchmarks bloaty_fuzz_target_52948c file_magic_fuzzer_2d5f85 grok_grk_decompress_fuzzer_9cd001 lcms_cms_transform_all_fuzzer_97d37d libaom_av1_dec_fuzzer_6e1848 mbedtls_fuzz_dtlsclient_7c6b0e sqlite3_ossfuzz_228a52

vanhauser-thc · 2023-07-28T12:42:05Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-07-28-aflpp --fuzzers aflplusplus aflplusplus_func aflplusplus_loop aflplusplus_loopfunc

vanhauser-thc · 2023-08-13T10:00:07Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-08-13-aflpp --fuzzers aflplusplus aflplusplus_paths aflplusplus_407

vanhauser-thc · 2023-08-13T10:40:50Z

@alan32liu @jonathanmetzman hmm seems like something is broken, the experiment is not starting :(

DonggeLiu · 2023-08-13T23:34:33Z

Yeah strange, the gcb CI did not even show up.
Let me re-run the command in case the error is flaky.

Update: It does not seem to work.

DonggeLiu · 2023-08-13T23:35:24Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-08-13-aflpp --fuzzers aflplusplus aflplusplus_paths aflplusplus_407

DonggeLiu · 2023-08-17T03:42:01Z

Let me close and reopen this to check if the GCB issue has been fixed.

DonggeLiu · 2023-08-17T03:42:29Z

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-08-13-aflpp --fuzzers aflplusplus aflplusplus_paths aflplusplus_407

DonggeLiu · 2023-08-17T03:42:54Z

It seems working : )

vanhauser-thc · 2023-08-17T05:37:11Z

Hmm thrree is no report though yet

DonggeLiu · 2023-08-17T10:53:04Z

It seems to be available now : )
The experiment data.
The experiment report.

two new experiments

bfea414

vanhauser-thc enabled auto-merge (squash) June 21, 2023 09:22

dummy change

0dab927

vanhauser-thc disabled auto-merge June 21, 2023 12:07

new experiment

76d436f

txt variants

ab97217

another experiment

96c5177

new experiment

779ed2a

fix build for coverage benchmarks

b5cd2ca

loopfunc

78fdcea

vanhauser-thc added 2 commits August 13, 2023 11:58

new variant

73f3544

update afl++ commit id

95fc8f9

DonggeLiu closed this Aug 17, 2023

DonggeLiu reopened this Aug 17, 2023

This was referenced Aug 17, 2023

Request an experiment for aflpp_symsan #1884

Closed

add new fuzzer: sievefuzz #1881

Open

Merge branch 'google:master' into master

f1e5fa3

vanhauser-thc closed this Aug 24, 2023

This was referenced Sep 25, 2023

Update LibAFL #1902

Merged

Orchesctra: A Collaborative Fuzzer Framework #1904

Closed

DonggeLiu mentioned this pull request Oct 5, 2023

libafl_libfuzzer optimization experiment #1903

Closed

DonggeLiu mentioned this pull request Apr 22, 2024

Adding muttfuzz and requesting an experiment #1967

Open

DonggeLiu mentioned this pull request Aug 23, 2024

Experiment request for custom benchmarks #2026

Closed

DonggeLiu mentioned this pull request Jan 23, 2025

Adding a new fuzzer #2069

Closed

Conversation

vanhauser-thc commented Jun 21, 2023

Uh oh!

vanhauser-thc commented Jun 21, 2023

Uh oh!

DonggeLiu commented Jun 21, 2023

Uh oh!

DonggeLiu commented Jun 21, 2023

Uh oh!

vanhauser-thc commented Jun 21, 2023

Uh oh!

DonggeLiu commented Jun 21, 2023

Uh oh!

DonggeLiu commented Jun 21, 2023

Uh oh!

vanhauser-thc commented Jun 23, 2023

Uh oh!

vanhauser-thc commented Jun 23, 2023

Uh oh!

vanhauser-thc commented Jun 24, 2023

Uh oh!

DonggeLiu commented Jun 25, 2023

Uh oh!

DonggeLiu commented Jun 25, 2023

Uh oh!

DonggeLiu commented Jun 26, 2023

Uh oh!

DonggeLiu commented Jun 26, 2023

Uh oh!

vanhauser-thc commented Jun 28, 2023

Uh oh!

DonggeLiu commented Jun 29, 2023

Uh oh!

vanhauser-thc commented Jun 29, 2023

Uh oh!

DonggeLiu commented Jun 29, 2023

Uh oh!

DonggeLiu commented Jun 29, 2023

Uh oh!

vanhauser-thc commented Jun 30, 2023

Uh oh!

DonggeLiu commented Jun 30, 2023

Uh oh!

DonggeLiu commented Jun 30, 2023

Uh oh!

vanhauser-thc commented Jun 30, 2023

Uh oh!

DonggeLiu commented Jul 20, 2023

Uh oh!

vanhauser-thc commented Jul 21, 2023

Uh oh!

vanhauser-thc commented Jul 21, 2023

Uh oh!

vanhauser-thc commented Jul 27, 2023

Uh oh!

vanhauser-thc commented Jul 28, 2023

Uh oh!

vanhauser-thc commented Aug 13, 2023

Uh oh!

vanhauser-thc commented Aug 13, 2023

Uh oh!

DonggeLiu commented Aug 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DonggeLiu commented Aug 13, 2023

Uh oh!

DonggeLiu commented Aug 17, 2023

Uh oh!

DonggeLiu commented Aug 17, 2023

Uh oh!

DonggeLiu commented Aug 17, 2023

Uh oh!

vanhauser-thc commented Aug 17, 2023

Uh oh!

DonggeLiu commented Aug 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

DonggeLiu commented Aug 13, 2023 •

edited

Loading

DonggeLiu commented Aug 17, 2023 •

edited

Loading