Encoder Progress / Source Progress

[alvises]

It would be nice to have a way to know the progress of compression and, in case of :url entries download progress as well.

The two options I have in mind are:

• be notified asynchronously via message, so we pass a pid to Packmatic.build_stream/2 in the options.
• or synchronously, passing a callback.

[evadne]

For progress of the Encoder, reporting can be had by integrating with gen_event, although some custom structs will be defined to represent the possible range of events. For progress of each Source, that may have to be done as an opt-in thing as support for custom sources has now been implemented and will come up soon. Further thinking required.

[evadne]

I have kind of sorted this out for the Encoder but need more design for Sources to participate.

[evadne]

@alvises Please take a look at #5 esp. test/packmatic/event_test.exs and let me know if that is more or less aligned with what we want to do here.

As to Source progress, since not all Sources know how many bytes they need to read (sometimes you don’t have this information, for example if you are reading from a connection that is serving a Packmatic stream 😛), I am thinking of something optional. For example, the signature of c:Packmatic.Source.init/1 and/or c:Packmatic.Source.read/1 could be extended to allow some sort of metadata (e.g. estimated bytes total/remaining) to be returned.

[alvises]

handler_fun = fn event ->
  :ok = Process.send(parent, event, [:noconnect, :nosuspend])
end

Packmatic.build_stream(on_error: :skip, on_event: handler_fun)

Really nice! Exactly what I meant :)

[evadne]

@alvises 👋

This is good news, I shall merge this soon.

[alvises]

@alvises 👋

This is good news, I shall merge this soon.

That's great!!

[evadne]

This is merged into develop and a new release on Hex will be coming tomorrow.