FileStore

FileStore allows you to read, write, upload, download, and interact with files, regardless of where they are stored.

It includes adapters for the following storage backends:

• Disk
• S3
• Memory
• Null

View the documentation

Installation

The package can be installed by adding file_store to your list of dependencies in mix.exs:

def deps do
  [{:file_store, "~> 0.3"}]
end

Usage

Configure a new store:

store = FileStore.Adapters.Disk.new(
  storage_path: "/path/to/store/files",
  base_url: "http://example.com/files/"
)

Now, you can manipulate files in your store:

iex> FileStore.upload(store, "hello.txt", "world.txt")
:ok

iex> FileStore.read(store, "world.txt")
{:ok, "hello world"}

iex> FileStore.stat(store, "world.txt")
{:ok,
 %FileStore.Stat{
   etag: "5eb63bbbe01eeed093cb22bb8f5acdc3",
   key: "hello.txt",
   size: 11,
   type: "application/octet-stream"
 }}

iex> FileStore.get_public_url(store, "world.txt")
"http://localhost:4000/world.txt"

Click here to see all available operations.

Middleware

Logger

To enable logging, just wrap your store with the logging middleware:

iex> store
...> |> FileStore.Middleware.Logger.new()
...> |> FileStore.read("test.txt")
# 02:37:30.724 [debug] READ OK key="test.txt"
{:ok, "hello"}

Errors

All adapters return a consistent error struct. Pattern-match on the category you care about:

case FileStore.read(store, "bizcorp.jpg") do
  {:ok, data}                         -> data
  {:error, %FileStore.NotFound{}}     -> nil
  {:error, %FileStore.PermissionDenied{}} -> reraise_as_403()
  {:error, %FileStore.Network{}}      -> retry()
  {:error, error}                     -> raise error
end

Available error structs (all implement Exception, so you can raise them):

• FileStore.NotFound — key does not exist (:enoent, HTTP 404, etc.)
• FileStore.PermissionDenied — caller lacks access (:eacces, HTTP 403, etc.)
• FileStore.Conflict — resource state conflicts (:eisdir, HTTP 409, etc.)
• FileStore.InvalidArgument — bad input (:invalid_tags, HTTP 400, etc.)
• FileStore.Network — transport failure (connection reset, HTTP 5xx, etc.)
• FileStore.Error — catch-all for unclassified errors

Every struct carries :reason (the raw underlying cause), :operation, and relevant context fields (:key, :src, :dest, :path, :tags, :acl).

Retry on network failure:

def with_retries(fun, attempts \\ 3) do
  case fun.() do
    {:error, %FileStore.Network{}} when attempts > 1 ->
      Process.sleep(backoff(attempts))
      with_retries(fun, attempts - 1)
    other ->
      other
  end
end

Prefix

The prefix middleware allows you to prepend a prefix to all operations.

iex> store
...> |> FileStore.Middleware.Prefix.new(prefix: "company/logos")
...> |> FileStore.read("bizcorp.jpg")

In the example above, bizcorp.jpg was translated to companies/logos/bizcorp.jpg.

Creating a store

You can also create a dedicated store in your application.

defmodule MyApp.Storage do
  use FileStore.Config, otp_app: :my_app
end

You'll need to provide configuration for this module:

config :my_app, MyApp.Storage,
  adapter: FileStore.Adapters.Null,
  middleware: [FileStore.Middleware.Errors]

Now, you can interact with your store more conveniently:

iex> MyApp.Storage.write("foo", "hello world")
:ok

iex> MyApp.Storage.read("foo")
{:ok, "hello world"}

Contributing

In order to run the test suite, you'll need Docker and Docker Compose. Docker is used to run services like Minio locally, so that we can integration test the S3 adapter.

To install dependencies and start services:

$ bin/setup

Run the test suite:

$ bin/test