Slap.Streams.Cluster (slap_streams v0.3.0)

Copy Markdown View Source

The shards that hold the streams, a Slap.Cluster. Each shard allocates stream ids and supervises one process per active stream.

Run one instance of this built-in cluster per VM. It can own many shards and participate in a cluster across VMs. Slap.Streams.Metrics reports node-wide metrics for this instance.

To configure a cluster under your application's name, define a module with use Slap.Streams.Cluster, otp_app: :my_app, supervise that module in place of Slap.Streams.Cluster, and pass cluster: MyApp.StreamsCluster to Streams operations and Slap.Streams.HTTP.Router. Run one Streams cluster per VM; metrics and the router's request-body budget are node-wide.

Configure it like any Slap.Cluster:

config :slap_streams, Slap.Streams.Cluster,
  store: {:url, "s3://streams/db", aws_endpoint: "https://rustfs:9000"},
  shards: 8

Durable appends wait for SlateDB's next WAL flush. This cluster sets flush_interval to 10 ms, the value used by the Durable Streams design; SlateDB's default is 100 ms. Other settings leave the 10 ms default intact. The README describes the latency and object-store request trade-off.

Service settings go in child_options: at start. See child_option_keys/0 for the accepted keys. Values are positive integers in milliseconds for timeouts and intervals, bytes for byte limits, and entries for :deleter_page. :max_fork_copy_bytes and :fork_copy_grace may also be zero.

Summary

Functions

Defines a Streams cluster under the given :otp_app.

Where every shard is. See Slap.Cluster.assignments/1.

Runs a function with a shard's context. See Slap.Cluster.call/4.

Returns the accepted :child_options keys for a Streams cluster.

The shards open on this node.

Where a shard is. See Slap.Cluster.lookup/2.

The shard for a placement key. See Slap.Cluster.shard_for/2.

Starts the cluster. opts override the application environment, which overrides the :defaults given to use Slap.Cluster.

Stops the cluster on this node, closing its shards.

A name for a per-shard process. See Slap.Cluster.via/3.

Functions

__using__(opts)

(macro)

Defines a Streams cluster under the given :otp_app.

assignments()

Where every shard is. See Slap.Cluster.assignments/1.

call(shard, mfa, opts \\ [])

Runs a function with a shard's context. See Slap.Cluster.call/4.

child_option_keys()

@spec child_option_keys() :: [atom()]

Returns the accepted :child_options keys for a Streams cluster.

local_shards()

The shards open on this node.

lookup(shard)

Where a shard is. See Slap.Cluster.lookup/2.

shard_for(key)

The shard for a placement key. See Slap.Cluster.shard_for/2.

start_link(opts \\ [])

Starts the cluster. opts override the application environment, which overrides the :defaults given to use Slap.Cluster.

stop(timeout \\ :infinity)

Stops the cluster on this node, closing its shards.

via(shard, name)

A name for a per-shard process. See Slap.Cluster.via/3.