Upgrade base images, benchmark images, and Python. by DonggeLiu · Pull Request #1526 · google/fuzzbench

DonggeLiu · 2022-10-14T01:52:11Z

I guess this branch has what we need? Not sure why it does not has a PR though.
I make it up-to-date with the master branch so that we can discuss here.

Related: google/oss-fuzz#6180

DonggeLiu · 2022-10-14T06:17:47Z

A quick update:
I am working on merging modifications from other PR to here.
Currently, I have a docker/base-image/Dockerfile that is based on focal and has python3.10.8 (the latest version).
I also updated the pip packages in requirements.txt to the latest version and managed to install them successfully in docker/base-image/Dockerfile.
I will push later.

DonggeLiu · 2022-10-14T06:22:51Z

TODOs in this PR:

Check if there is any conflict with the new version of Ubuntu, python, and pip packages (like Update ubuntu to jammy and python to 3.10 #1521).
Check if this PR missed anything in Upgrade to python3.10 #1493.
Use a newer OSS-Fuzz docker image (like upgrade images #1441).
Final check for conflicts.

DonggeLiu · 2022-10-14T06:24:01Z

TODOs in future PRs:

Fix new lint/formatter issues, as we are using a newer version of pylint now. WIP at Fix lint #1529.
Update documentation, done in Update documentation #1533.

jonathanmetzman · 2022-10-14T10:56:06Z

btw you can just checkout https://github.com/google/fuzzbench/pull/1441/files locally and start pushing to that PR. I don't think we need another one.

jonathanmetzman · 2022-10-14T10:57:03Z

    - 'parent_image=gcr.io/fuzzbench/builders/coverage/{benchmark}-intermediate'
  depends_on:
    - 'coverage-{benchmark}-builder-intermediate'
-    - 'base-image'


What's the reason for this removal?

What's the reason for this removal?

I do not know, it comes with this branch of yours : )

Would you prefer me to revert this?

Yeah lets rever this

I think this is causing the CI failure as we discussed

DonggeLiu · 2022-10-17T05:05:33Z

CI failed because we've updated pylint.

DonggeLiu · 2022-10-18T06:24:14Z

I've fixed all meaningful lint errors in #1529, which involves more than 60+ files.
There are many lint errors I consider not worth fixing and plan to let pylint ignore them, could you please take a look over there and let me know if you'd like me to fix them?

Once that PR is ready, I will merge its branch here.

DonggeLiu · 2022-11-22T06:27:32Z

It is OK to let presubmit fail for now: I will bring in the PR that fixes it at the end.

Some projects use `-Werror` to turn all warnings into errors. This affects `Centipede` as we do not separate build and linking flags as it expects, which leads to `unused-command-line-argument` warnings. This PR disables turning that specific warning into errors and keeps the rest the same. See [the same PR from OSS-Fuzz](google/oss-fuzz#9030) for more info and [the error in this PR](google/oss-fuzz#8990) for its use case.

This PR fixes new lint errors from the latest `pylint`, which is required as #1526 upgrades `Python` to `3.10`. I reckon I have fixed all meaningful errors in the *`FuzzBench` framework*, not sure if the rest is worth fixing or could be ignored.

DonggeLiu · 2022-11-25T06:20:30Z

@jonathanmetzman : Please let me know if Fix test_build_all_measurers makes sense.
I was unsure if I completely understood the previous code (which failed the unittest), so I was trying to fix it based on its function description.

DonggeLiu · 2022-11-25T06:23:56Z

Not sure what is the best way to fix validate_fuzzers_and_benchmarks in presubmit:
I guess we don't want to delete the temporary fuzzers and variations now, but also do not want to always check them in CIs.

That is also the only thing left to pass all CI tests.

DonggeLiu · 2022-12-01T00:10:24Z

Not sure what is the best way to fix validate_fuzzers_and_benchmarks in presubmit: I guess we don't want to delete the temporary fuzzers and variations now, but also do not want to always check them in CIs.

That is also the only thing left to pass all CI tests.

@jonathanmetzman : A gentle ping on this : )
Would now be a good time to delete the temporary fuzzers or remove them from CIs, so that we can merge this PR to master?
Also, I noticed that there are experiments running for weeks (r-2022-11-11-um-prioritize-2209607) or months (r-2022-10-13-um-final-3-2201231), is that intended?
Thanks!

DonggeLiu · 2022-12-02T01:10:35Z

jonathanmetzman · 2022-12-02T19:03:19Z

@jonathanmetzman: Here is the full list of current fuzzers, could you please also tick the ones that you'd like to keep? Thanks! I will delete the rest.

We need to keep coverage for sure.
Let's do the upgrade and then delete in a subsequent PR (OK if CI fails)
Explain why we need to delete some of these please. Is it because they need to be fixed to pass linting?
Anyway, I've marked some I think we should keep

DonggeLiu · 2022-12-02T22:40:49Z

We need to keep coverage for sure. Let's do the upgrade and then delete in a subsequent PR (OK if CI fails)

Sure! I will merge this on Monday, then.

Explain why we need to delete some of these please. Is it because they need to be fixed to pass linting?

Yep, it is mainly to avoid the validate_fuzzers_and_benchmarks error from presubmit.

FuzzBench now works on Ubuntu 20.04 and Python 3.10. Co-authored-by: Jonathan Metzman <metzman@chromium.org>

jonathanmetzman and others added 2 commits August 9, 2021 08:37

Upgrade Ubuntu version to focal (20.04)

0c4570a

Related: google/oss-fuzz#6180

Merge branch 'master' into upgrade-base-image

0c7c3ad

DonggeLiu requested a review from jonathanmetzman October 14, 2022 01:52

Some additional fixes to upgrade Ubuntu

40ef5bc

jonathanmetzman reviewed Oct 14, 2022

View reviewed changes

DonggeLiu added 3 commits October 17, 2022 15:54

Update python (to 3.10.4), packages, and implementation

57481ae

Install google-cloud-cli

c068796

Remove png (from an existing commit)

1c3b0fe

DonggeLiu mentioned this pull request Oct 19, 2022

Fix lint #1529

Merged

DonggeLiu added 3 commits October 19, 2022 16:20

Remove redundant python&pip install

7908284

Manually merging #1493 to this branch

72c16df

Manually merging #1441

6b536a1

DonggeLiu requested a review from jonathanmetzman October 19, 2022 09:34

DonggeLiu added 5 commits October 20, 2022 11:27

Remove redundant print

b22a15f

Merge branch 'master' into upgrade-base-image

33f1757

Merge branch 'master' into upgrade-base-image

8f16bdd

Update python version

071f98e

Update ubuntu version

8c07672

DonggeLiu mentioned this pull request Oct 20, 2022

Update documentation #1533

Closed

Merge branch 'update_documentation' into upgrade-base-image

0e8793d

DonggeLiu changed the title ~~Upgrade base image to 20.04~~ Upgrade base images, benchmark images, and Python. Oct 21, 2022

DonggeLiu added 3 commits October 21, 2022 16:13

Install Python3.10 in BenchmarkBuilder

6a8ad4b

Merge branch 'master' into upgrade-base-image

b81472d

Upgrade python in files that missed earlier

d9142e8

Merge branch 'master' into upgrade-base-image

a1307ea

DonggeLiu added 9 commits November 23, 2022 14:48

Benchmark popper does not support wingfuzz

e87097d

Remove wingfuzz from systemd

9e67151

Fix lint (#1529)

2ba4957

This PR fixes new lint errors from the latest `pylint`, which is required as #1526 upgrades `Python` to `3.10`. I reckon I have fixed all meaningful errors in the *`FuzzBench` framework*, not sure if the rest is worth fixing or could be ignored.

Merge branch 'master' into upgrade-base-image

aee441d

Fix test_build_all_measurers

63f0eaa

Fix typing hint

a011d7a

Temporarily convert bug/oss-fuzz benchmarks to code

ce36cc9

Update libpng's name in tests

3d9c7f1

Update the new list of core fuzzers

aa904aa

DonggeLiu added 3 commits December 3, 2022 09:47

Merge branch 'master' into upgrade-base-image

5e03dc9

Reorganise fuzzer list

042ee4f

Merge branch 'master' into upgrade-base-image

92504da

DonggeLiu merged commit a6a5a9a into master Dec 5, 2022

DonggeLiu deleted the upgrade-base-image branch December 5, 2022 00:27

This was referenced Dec 5, 2022

Fix new presubmit error #1566

Merged

Update ubuntu to jammy and python to 3.10 #1521

Closed

Upgrade to python3.10 #1493

Closed

upgrade images #1441

Closed

Deprecate fuzzzers and ignore fuzzers from CI #1570

Open

DonggeLiu mentioned this pull request Aug 3, 2024

Prescientfuzz testing #1982

Closed

SPINO-11 pushed a commit to SPINO-11/green_fuzzing_evaluation that referenced this pull request Aug 9, 2025

Upgrade base images, benchmark images, and Python. (google#1526)

5d93c12

FuzzBench now works on Ubuntu 20.04 and Python 3.10. Co-authored-by: Jonathan Metzman <metzman@chromium.org>

Conversation

DonggeLiu commented Oct 14, 2022

Uh oh!

DonggeLiu commented Oct 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DonggeLiu commented Oct 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DonggeLiu commented Oct 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jonathanmetzman commented Oct 14, 2022

Uh oh!

jonathanmetzman Oct 14, 2022

Choose a reason for hiding this comment

Uh oh!

DonggeLiu Oct 15, 2022

Choose a reason for hiding this comment

Uh oh!

DonggeLiu Oct 20, 2022

Choose a reason for hiding this comment

Uh oh!

jonathanmetzman Oct 20, 2022

Choose a reason for hiding this comment

Uh oh!

jonathanmetzman Oct 27, 2022

Choose a reason for hiding this comment

Uh oh!

DonggeLiu commented Oct 17, 2022

Uh oh!

DonggeLiu commented Oct 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DonggeLiu commented Nov 22, 2022

Uh oh!

DonggeLiu commented Nov 25, 2022

Uh oh!

DonggeLiu commented Nov 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DonggeLiu commented Dec 1, 2022

Uh oh!

DonggeLiu commented Dec 2, 2022 • edited by jonathanmetzman Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jonathanmetzman commented Dec 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DonggeLiu commented Dec 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

DonggeLiu commented Oct 14, 2022 •

edited

Loading

DonggeLiu commented Oct 14, 2022 •

edited

Loading

DonggeLiu commented Oct 14, 2022 •

edited

Loading

DonggeLiu commented Oct 18, 2022 •

edited

Loading

DonggeLiu commented Nov 25, 2022 •

edited

Loading

DonggeLiu commented Dec 2, 2022 •

edited by jonathanmetzman

Loading

jonathanmetzman commented Dec 2, 2022 •

edited

Loading

DonggeLiu commented Dec 2, 2022 •

edited

Loading