Filesystem

The filesystem connector watches a directory and submits each matching file to a flow. It is a separate process from the platform, configured from the same .flow.md files, so a flow’s trigger lives next to the flow it triggers.

$ nexus-vfs-connector --server http://localhost:9090 order-intake.flow.md

Configuring a trigger

A flow declares its trigger in front matter. The connector reads the file, finds the configuration and starts a worker; a flow without it is ignored.

---
flowmarkdown_version: "0.1"
flow: order-intake
tenant: acme
trigger_type: vfs
trigger_path: /var/spool/orders/in
trigger_done_path: /var/spool/orders/done
trigger_error_path: /var/spool/orders/error
trigger_pattern: '.*\.xml$'
trigger_poll_interval_ms: 5000
max_message_bytes: 1048576
trigger_move_timestamp: true
---

Key	Meaning
`trigger_type`	Must be `vfs` for this connector to pick the flow up
`trigger_path`	Directory to watch
`trigger_done_path`	Where a file goes after it has been submitted
`trigger_error_path`	Where a file goes on failure. Optional
`trigger_pattern`	Regular expression matched against the file name
`trigger_poll_interval_ms`	How often the directory is scanned
`max_message_bytes`	Largest file accepted. `trigger_max_body_bytes` is a deprecated alias
`trigger_move_timestamp`	Append a timestamp to the file name when moving it

trigger_pattern is a regular expression, not a shell glob. .*\.xml$ matches XML files; *.xml does not do what you expect.

Connector options

$ nexus-vfs-connector \
    --server http://localhost:9090 \
    --store /var/lib/nexus/vfs-connector.db \
    order-intake.flow.md invoice-intake.flow.md

Flag	Default	Environment variable
`--server`	`http://localhost:9090`	`NEXUS_SERVER_URL`
`--store`	`nexus-vfs-connector.db`	—
`--api-key`	none	`NEXUS_API_KEY`
(positional)	required	one or more `.flow.md` paths

Each flow file gets its own worker, so one process can serve several directories.

The store is a local SQLite database used for crash recovery. Give it a stable path — a store in the working directory means recovery state depends on where you launched the process from.

How a file is processed

On each poll, the connector lists the directory, keeps the files matching the pattern, and sorts them by modification time. Files are then processed oldest first.

For each file:

Compute its identity and check whether it has already been processed.
If it is over the size limit, record it as refused, move it to trigger_error_path if one is configured, and skip it. The file is never read into memory, and it is never filed under done.
Record the submission in the store.
Report a connector_received event to the platform.
Submit the content to the flow’s queued endpoint.
Move the file to trigger_done_path.
Mark it done in the store.

The content type is chosen from the file extension:

Extension	Content type
`.xml`	`application/xml`
`.json`	`application/json`
`.csv`	`text/csv`
anything else	`application/octet-stream`

Because submission goes to the queued endpoint, an XML file arrives at the flow flattened — child elements become object fields and the root tag is dropped. Write the flow’s expressions accordingly; see JSON and XML.

File identity, and what it means for duplicates

A file is identified by path:mtime:size. All three participate, which has consequences worth knowing:

The same file re-created with different content is a different identity, so it is processed again. That is usually what you want for a spool directory.
A file touched without a content change gets a new mtime, so it is a new identity and is processed again.
Two files with the same name in different directories are distinct.
Two different files that happen to share a path, mtime and size cannot occur, since the path is unique at a point in time.

Note that mtime is taken at second granularity. A file replaced within the same second, at the same path and the same size, is not distinguishable from the original and will be skipped.

The delivery guarantee

At least once. A message can be delivered twice, and there is a specific window where it will be.

Look at the order above: the file is submitted (step 5), then moved (step 6), then marked done (step 7). A crash between 5 and 7 leaves the store’s record in the pending state, so on restart replay_pending submits it again.

Crash point	Result on restart
Before step 3	Nothing recorded; the file is still in the watch directory and is processed normally
Between 3 and 5	Recorded as pending; resubmitted. May be a first delivery or a duplicate — the connector cannot tell whether the submission reached the platform
Between 5 and 6	Recorded as pending, file still in place; resubmitted — a duplicate
Between 6 and 7	Recorded as pending, file already moved; resubmitted — a duplicate
After 7	Nothing to do

So flows fed by this connector must tolerate the same file arriving twice. Derive any downstream idempotency key from the file’s content or from a field inside it — not from uuid(), which produces a new value on every attempt.

This is the same guarantee the queue itself gives, so a flow written to be safe under queue redelivery is already safe here.

Permanent refusals

The connector retries a failed submission — but only where retrying can work. Network failures, 429 and 5xx are retried; 401, 403 and 413 are final answers and end the attempt.

Status	What it means	What happens to the file
`401`, `403`	The key is wrong, revoked, expired, or not allowed on this flow or tenant	The file is left exactly where it is and the worker stops. Fix the key, restart, and the file is submitted then.
`413`	The file is over the platform’s ceiling for this flow	Moved to `trigger_error_path` if configured, and recorded as refused either way. The worker carries on with the next file.
`429`	Rate limited	Waits for `Retry-After` when the server sends one (seconds form, capped at 5 minutes), otherwise backs off, then retries
`5xx`, network	The platform is down or restarting	Backs off and retries

Stopping the retry loop is only half of it. A refused file stays in the watch directory, so the next poll would pick it up again and start over. That is why a refusal is also recorded: with no error directory configured, the record in the connector’s store is the only thing that keeps the file out of later scans. Every permanent refusal is logged at error with the status, tenant, flow and what to do about it — a file that can be neither sent nor moved must not pass silently.

A refused file never goes to trigger_done_path. That directory is what an operator reads as “delivered”.

Not covered: 400, 404 and 422 are still retried indefinitely. 404 is why — a connector started before its flow is deployed gets one legitimately, and there retrying is the right answer.

Reporting

The connector reports two event kinds to the platform, both carrying a correlation ID so a file can be followed from arrival to execution:

Event	Meaning
`connector_received`	A file was accepted and is being submitted
`connector_oversized`	A file exceeded `max_message_bytes` and was skipped

Follow one file through the structured log by its correlation ID.

A complete setup

$ mkdir -p /var/spool/orders/{in,done,error}

order-intake.flow.md:

---
flowmarkdown_version: "0.1"
flow: order-intake
tenant: acme
effects: [http_egress]
trigger_type: vfs
trigger_path: /var/spool/orders/in
trigger_done_path: /var/spool/orders/done
trigger_error_path: /var/spool/orders/error
trigger_pattern: '.*\.xml$'
trigger_poll_interval_ms: 5000
max_message_bytes: 1048576
trigger_move_timestamp: true
queue_max_attempts: 5
---

Picks up order files from the spool directory and forwards them.

## Step: check
```validate
$.orderId != null | "orderId is required" | syntactic
```

## Step: forward
effects: [http_egress]
endpoint: https://orders.example.com/ingest
method: POST
content_type: application/json

$ nexus tenant create --id acme --display-name "Acme Ltd"
$ nexus deploy order-intake.flow.md --version 1.0.0
$ nexus keys create --tenant acme --label vfs-connector \
                    --scopes '*' --flows order-intake
created  API key id=1
  key:   nxk_4f19c0b27ad3489e93f5c1e6a70d2b84

$ nexus serve --port 9090 &
$ export NEXUS_API_KEY=nxk_4f19c0b27ad3489e93f5c1e6a70d2b84
$ nexus-vfs-connector --server http://localhost:9090 \
                      --store /var/lib/nexus/vfs.db \
                      order-intake.flow.md &
$ cp order-A1.xml /var/spool/orders/in/

The connector authenticates like any other caller — there is no connector-shaped exemption. Give it its own key, scoped with --flows to the flow it submits to. Without one it gets 401 on its first submission and stops rather than retrying: a credential refusal is terminal, and the file stays where it is.

Within one poll interval the file is submitted, moved to done with a timestamp appended, and the flow runs.

Not supported

There is no watch based on filesystem notifications — the directory is polled. Subdirectories are not traversed. A file that fails processing is not retried by the connector; retries are the queue’s job, and the dead-letter queue is where an exhausted message ends up.