Several Dagster features, like schedules, sensors, and run queueing, require a long-running dagster-daemon process to be included with your deployment.
To start the dagster-daemon process locally, launch the following command in the same folder as your workspace.yaml file and keep the process running:
$ dagster-daemon run
Check the Deployment Guides section for more information on how to deploy the daemon in other environments, like Docker or Kubernetes.
The dagster-daemon process reads from your Dagster instance file to determine which daemons it should include. Each of those daemons then runs on a regular interval in its own threads. Daemons that need to load your workspace will periodically reload the workspace from your workspace.yaml file. If you change your workspace.yaml file, you do not need to reload the dagster-daemon process in order for those changes to be included.
The following daemons are currently available:
The scheduler daemon is responsible for creating runs from any schedules that are turned on. This daemon will run as long as you have not overridden the default DagsterDaemonScheduler as the scheduler on your instance.
The run queue daemon is responsible for launching queued runs, taking into account any limits and prioritization rules you've set on your instance. You can enable this daemon by setting the run coordinator on your instance to QueuedRunCoordinator.
The sensor daemon is responsible for creating runs from any sensors that are turned on. This daemon is always enabled.
The run monitoring daemon is responsible for handling run worker failures. This daemon can be enabled with the run_monitoring field in your instance.
To check the status of your dagster-daemon process within Dagit, click on "Status" in the left nav. This will take you a page where you can see information about each daemon that's currently configured on your instance.
Each daemon periodically writes a heartbeat to your instance storage, so if a daemon doesn't show a recent heartbeat on this page, it likely indicates that you should check the logs from your dagster-daemon process for errors.