Extend marmoset to provide callback endpoint for the installimage

[bastelfreak]

We've got two reporting functions in our installimage:

• after an installation
• after a successful boot after an installation

We need to extend the installimage endpoint or create a new one which the installimage itself can call. In the future we want to report statistics as well (used installimage configuration). The installimage will pass the UUID as an identifer. If a successful boot was reported we could remove the installimage config. The reporting endpoint needs to be configureable via our configuration file to turn it on/off. We shouldn't require any database stuff or similar if it is turned off.

Open questions:

• do we only rely on the UUID for authentication?
• what to do with requests from installations without UUID?
• which databaseengine do we want to use?

[bastelfreak]

@sebastianrakel I would love to hear your opinions here.

[bastelfreak]

we won't use sqlite because that will horribly fail if someone uses API + CLI at the same time.

[pbyatshon]

if we use the sqlite locking mechanism properly and dont use NFS or FAT as underlying filesystem, there should be no problem with concurrent accesses, even from different processes. So sqlite might still be an option if we want to keep it slim.

For further details:
https://www.sqlite.org/lockingv3.html
https://www.sqlite.org/wal.html

[bastelfreak]

Anybody has experiences with largs sqlite databases? The schema is hopefully pretty simple, but I guess we want to save information on a longer time which will result in a huge amount of data? It feels like postgres would be the better solution.

[pbyatshon]

i added a new feature branch for the possible integration of an installationstatus endpoint with sqlite as db backend. Any proposals are welcome.

What definetly needs to be clarified: What should be included in a status update? Date, status code and a status message. Anything else?

[pbyatshon]

@bastelfreak i tested with 1mio rows, couldn't recognize any problems, also when stressing the db hard with inserting thousands of entries while reading the db. Probably we won't get 10k q/s, but i guess that won't be required for marmoset ;).

[bastelfreak]

okay, sqlite will probably work. also switching to some different (R)DBMS shouldnt be that hard.

[pbyatshon]

yes, switching to a "real" database won't be much effort. I would prefer integration of an ORM like SQLAlchemy, too, but didn't used one yet, so i did it the easy way first.

Furthermore there is no relation in the database yet, just one table. We'll have to see, what we want to add as further information. For tests i created a uuid -> id mapping table and removed the uuid from the installstatus table, but due to the lack of noticeable performance increasement with sqlite, i kept it simple and inserted all to one table.

[bastelfreak]

This got implemented in #87 and #66