matze/farmOS-dev
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

clean install

Browse Source
This commit is contained in:
matze
2024-12-10 15:08:16 +01:00
commit e14eb2d8fd
31193 changed files with 3555714 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

254
web/profiles/farm/docs/hosting/composer.md Normal file
View File

@@ -0,0 +1,254 @@
# Building farmOS with Composer
[Composer](https://getcomposer.org/) is the standard way to build and manage
PHP-based projects and their dependencies. farmOS uses Composer in its
automated build processes to generate packaged releases, including
[Docker](/hosting/install/#farmos-in-docker) images and
[tarballs](/hosting/install/#packaged-releases). These "official" releases
only include the core farmOS modules.
If you want to include additional modules that are created by yourself or
others in the community, you may find that you need more control over the
build process. This guide outlines the basic workflow for managing your
farmOS-based projects with Composer.
## Composer projects
First, it's important to understand the concept of a "Composer project".
If you are building a new PHP application from scratch with Composer, the
first step is to create a `composer.json` file. As described in the Composer
docs:
> To start using Composer in your project, all you need is a `composer.json`
> file. This file describes the dependencies of your project and may contain
> other metadata as well.
https://getcomposer.org/doc/01-basic-usage.md#composer-json-project-setup
### Project templates
Composer also provides a `create-project` command for starting a new project
from an existing one. This allows for "project templates" to be provided for
developers to build on top of.
> You can use Composer to create new projects from an existing package. This
> is the equivalent of doing a git clone/svn checkout followed by a
> `composer install` of the vendors.
https://getcomposer.org/doc/03-cli.md#create-project
A good example of this is Drupal's `recommended-project` template, which is
described as follows:
> These project templates serve as a starting point for creating a
> Composer-managed Drupal site. Once you have selected a template project and
> created your own site, you will take ownership of your new project files.
> Thereafter, future updates will be done with Composer.
https://www.drupal.org/docs/develop/using-composer/starting-a-site-using-drupal-composer-project-templates
For example, to create a new project based on Drupal's `recommended-project`
template:
composer create-project drupal/recommended-project
This will initialize a `composer.json` and `composer.lock` file that serves as
a template for a Drupal website.
### farmOS project template
farmOS provides its own project template:
https://packagist.org/packages/farmOS/project
The official farmOS releases are built from this template. It can also be used
to start your own project based on farmOS. This process is described in the
next section.
## Starting your farmOS project
To start your own project based on the farmOS project template, open a new
directory and run the following command:
composer create-project farmos/project:3.x-dev
This will create a `composer.json` file in your directory, copied directly from
the [farmOS project template](https://packagist.org/packages/farmos/project).
You can use this `composer.json` as the starting point for your own project's
code repository with [Git](https://git-scm.com/):
```shell
git init
git add composer.json
git commit -m 'Initial commit of my farmOS project.'
```
It is helpful to understand that your project is *not* farmOS, but rather
farmOS is a *dependency* of your project. You can see this clearly by observing
that `farmos/farmos` is included in your `composer.json` file's `require`
section. To customize your project, open `composer.json` in a text editor and
change the metadata (eg: `name`, `description`, `homepage`, etc).
### Building the codebase
To build your farmOS project's codebase, run the following command:
composer install --no-dev
This will do two very important things:
1. It will download all of your project's dependencies (including farmOS) and
build the file structure. At the end, you will have a fully-functional
codebase. Configure your web server's document root to point to the `web`
subdirectory, and you will be able to open farmOS in a browser.
2. It will create a `composer.lock` file that contains auto-generated
information about your project's dependencies (and their dependencies) with
the specific version that were downloaded. This `composer.lock` file should
be committed to source control. It ensures that building the project again
will always use the same versions of dependencies, for reproducible builds.
### Ignoring files
At this point there will be many files in your directory that should *not* be
committed to source control. Best practice is to ignore them with a `.gitignore` file.
Example `.gitignore`:
```
# Ignore Composer-generated files.
.editorconfig
.gitattributes
vendor
web/*
# farmOS uses this to store OAuth2 keys.
keys
# This creates an exception for ./web/modules/custom, which can be used for
# custom module code that is part of this repository.
!web/modules
web/modules/*
!web/modules/custom
```
### Community modules
farmOS community modules can be added to your project via Composer.
For example, this will add the
[farmOS Bee](https://www.drupal.org/project/farm_bee) module:
composer require drupal/farm_bee
Notice that `composer.json` and `composer.lock` are automatically updated.
Commit these changes to ensure the module will be downloaded every time your
codebase is built.
### Custom modules
The example `.gitignore` above includes a special rule to exclude the
`./web/modules/custom` directory. This allows you to include your own custom
modules within your project and commit them to source control.
For example, create a file called
`./web/modules/custom/mymodule/mymodule.info.yml`
with the following content:
```yaml
name: My module
description: A custom module for my farmOS project.
type: module
package: farmOS Custom
core_version_requirement: ^10
```
This can be committed to your project's Git repository, and installed in your
farmOS instance via the web UI or by running `drush en mymodule`.
## Updating dependencies
Composer provides a simple way to update project dependencies:
composer update --no-dev
composer update --no-dev
**Note: It is necessary to run this command twice to ensure all dependencies
are properly updated.** We have an issue open to figure out a better solution:
[Composer merge plugin dependencies are not correctly updated #653](https://github.com/farmOS/farmOS/issues/653).
This will check for newer versions of all your project's dependencies (based
on the version constraints in your `composer.json` file), install them, and
update your `composer.lock` file automatically.
It is important to run automated database updates and rebuild caches whenever
farmOS and/or dependency modules are updated. For this reason, it is good to
develop a deployment strategy that includes these steps whenever a new version
of your project's code is deployed. See [Updating farmOS](/hosting/update) for
more information.
### Pinning versions
The `farmos/farmos` dependency in the farmOS project template `composer.json`
defaults to `^3.0`, which means "the latest stable version of the 3.x branch".
You may want to pin this (or other dependencies) to a specific version so that
you can be very intentional with your upgrade process.
To pin the `farmos/farmos` dependency to a specific version (eg: `3.0.1`),
replace `^3.0` with `3.0.1` and run `composer update --no-dev`.
To update pinned dependencies, simply update the version in `composer.json` and
run `composer update --no-dev` again. Remember to run automated updates and
rebuild caches after deployment. See [Updating farmOS](/hosting/update) for
more information.
## Docker
Composer is only responsible for building the farmOS codebase. After that, it
is up to you to deploy it. One way to do this is with Docker.
As described above, the "official" farmOS Docker images are built using the
default farmOS project template. You can use a similar approach to build
custom Docker images with your custom codebase.
Example `Dockerfile`:
```Dockerfile
# Inherit from the upsteam farmOS 3.x image.
FROM farmos/farmos:3.x
# Install `jq` to help in extracting the farmOS version below.
RUN apt-get update && apt-get install -y jq
# Create a fresh /var/farmOS directory.
RUN rm -r /var/farmOS && mkdir /var/farmOS
# Copy composer.json and composer.lock into the image.
COPY composer.json /var/farmOS/composer.json
COPY composer.lock /var/farmOS/composer.lock
# Build the farmOS codebase with Composer as the www-data user in /var/farmOS
# with the --no-dev flag.
RUN (cd /var/farmOS; composer install --no-dev)
# Set the version in farm.info.yml to match the version locked by Composer.
# This is optional but is useful because the version will appear as the
# "Installation Profile" version at `/admin/reports/status` in farmOS.
RUN sed -i "s|version: 3.x|version: $(jq -r '.packages[] | select(.name == "farmos/farmos").version' /var/farmOS/composer.lock)|g" /var/farmOS/web/profiles/farm/farm.info.yml
# Copy the farmOS codebase into /opt/drupal.
RUN rm -r /opt/drupal && cp -rp /var/farmOS /opt/drupal
```
With the above example `Dockerfile` in your project's root directory, the
following commands will build a custom Docker image and run it with a volume
for the `sites` directory. Note that this example is not tailored for local
development, but can be used as a basis to design something that suits your
needs.
sudo docker build -t mycustomfarmos .
sudo docker run --rm -it -p 80:80 -v $(pwd)/sites:/opt/drupal/web/sites mycustomfarmos
See [Hosting farmOS in Docker](/hosting/install/#farmos-in-docker) for more
information.

32
web/profiles/farm/docs/hosting/email.md Normal file
View File

@@ -0,0 +1,32 @@
# Email configuration
farmOS needs to be able to send emails to users. This is used for password
reset emails, notifications, etc.
Depending on how you have farmOS deployed, there are a few ways to configure
your server to allow farmOS to send emails.
By default, farmOS will attempt to send emails via an SMTP server installed on
the same system. If you have [Postfix](http://www.postfix.org) installed, email
should work without any additional configuration, although they will most likely
be filtered as spam. [This StackOverflow topic](https://stackoverflow.com/questions/371/how-do-you-make-sure-email-you-send-programmatically-is-not-automatically-marked)
provides guidance for avoiding this.
## Docker
The [farmOS Docker images](/hosting/install#farmos-in-docker) do not include an
SMTP server, so you will see this error message when farmOS tries to send an
email:
> Unable to send e-mail. Contact the site administrator if the problem persists.
There are two potential solutions to this:
1. Install and configure the [SMTP](https://drupal.org/project/smtp) module.
This is a contributed Drupal module that allows emails to be relayed through
a third-party SMTP server. This module is not included with farmOS, but can
be downloaded into `[farmOS-codebase]/web/sites/all/modules` and enabled in
`https://[farmOS-hostname]/admin/modules`.
2. Create your own Docker image which inherits from the farmOS image. This
image can install an SMTP server like Postfix, which can be configured to
send email directly, or relay it through another SMTP server.

31
web/profiles/farm/docs/hosting/index.md Normal file
View File

@@ -0,0 +1,31 @@
# Hosting farmOS
farmOS is a web-based application that can be installed and hosted on a web
server, much like a website. This allows it to be accessed by multiple people
simultaneously, from any device with a web browser and internet connection.
## Self hosting
If you are familiar with hosting database-backed websites, you can install
farmOS on your own web server.
farmOS is [free software](https://en.wikipedia.org/wiki/Free_software), which
means you are free to download the code and host it yourself. You are also free
to modify and extend it to fit your needs.
For more information, including server requirements, refer to
[Installing farmOS](/hosting/install).
To understand how to update farmOS when a new version is released, refer to
[Updating farmOS](/hosting/update).
## Subscription hosting
If you are not comfortable hosting and maintaining your own server,
subscription hosting allows you to get started with farmOS quickly and easily.
[Farmier](https://farmier.com) is a subscription hosting service that provides
"*farmOS as a service*", including hosting, automatic updates, backups, SSL
security, and other features. Farmier was started by the creator of farmOS, and
subscriptions help to support the continued development of farmOS. For more
information, visit [https://farmier.com](https://farmier.com).

252
web/profiles/farm/docs/hosting/install.md Normal file
View File

@@ -0,0 +1,252 @@
# Installing farmOS
These instructions are for installing farmOS on a live production server.
For local development/testing, please refer to the
[development environment](/development/environment)
documentation.
## Server requirements
farmOS is based on [Drupal](https://drupal.org), and therefore shares many of
the same [requirements](https://drupal.org/docs/system-requirements).
### Web server
In addition to Drupal's basic requirements, farmOS has the following server
dependencies. The [farmOS Docker images](#farmos-in-docker) include these.
- **PHP 8+**
- **PHP configuration** - The following PHP settings are recommended:
- `memory_limit=256M`
- `max_execution_time=240`
- `max_input_time=240`
- `max_input_vars=5000`
- `realpath_cache_size=4096K`
- `realpath_cache_ttl=3600`
- **[PHP BCMath extension](https://www.php.net/manual/en/book.bc.php)** and
**[GEOS](https://trac.osgeo.org/geos)** are required for accurate geometric
calculations. farmOS can be installed without these, but production usage
without them is strongly discouraged.
### Database server
A database server needs to be provisioned that farmOS can connect to.
PostgreSQL is recommended. MySQL/MariaDB and SQLite are also supported.
This can be installed on the same server as farmOS (either directly or in a
Docker container), or it can be on a separate server.
If PostgreSQL is used, it must be version 12 or higher, and the `pg_trgm`
extension must be installed and enabled on the farmOS database. On PostgreSQL
13+ this will be enabled automatically by farmOS. On PostgreSQL 12, the
following SQL query must be run on the farmOS database by a PostgreSQL
superuser to enable the extension:
CREATE EXTENSION pg_trgm;
### SSL
Although not strictly a requirement, some features (like the "Geolocate" button
on maps) will only work over a secure connection. [Field Kit](https://farmOS.app)
requires SSL in order to connect to it. SSL is also recommended if you are
streaming sensor data into farmOS, to keep your sensor's private key a secret.
A common strategy is to use [Nginx](https://nginx.org) as a reverse proxy with
SSL termination, which listens on port 443 and forwards to farmOS on port 80.
[Let's Encrypt](https://letsencrypt.org) is a good option for free SSL
certificate issuance, and renewal can be automated via cron.
These resources may be helpful:
- [Drupal HTTPS Information](https://www.drupal.org/https-information)
- [Reverse Proxy Forum Post](https://farmos.discourse.group/t/running-behind-reverse-proxy/108) -
Includes links to related GitHub issues and examples of how others have
configured reverse proxies serving HTTPS.
- [Local HTTPS](/development/environment/https) - Documentation for running an
Nginx reverse proxy with self-signed certificates for local farmOS
development with HTTPS.
### Satellite map layers
farmOS includes an optional [Mapbox](https://www.mapbox.com) module that can be
enabled to add satellite imagery layers to the map. A Mapbox API key is
required. For more information, see Mapbox's official documentation:
[Access tokens](https://docs.mapbox.com/help/how-mapbox-works/access-tokens).
Enable the Mapbox module at Setup > Modules, and then add the API key at
Setup > Settings > Map > Mapbox.
## farmOS Codebase
There are two supported approaches to deploying the farmOS codebase:
1. Using [Docker](https://docker.com) images.
2. Using packaged releases.
Docker is the recommended method of hosting farmOS because it encapsulates the
server level dependencies that farmOS needs.
If you need to build a more customized farmOS codebase, including modules
provided by the community, or custom modules written by yourself, see
[Building farmOS with Composer](/hosting/composer).
### farmOS in Docker
Official farmOS Docker images are available on Docker Hub:
[https://hub.docker.com/r/farmos/farmos](https://hub.docker.com/r/farmos/farmos)
This allows farmOS to be run in a Docker container with:
docker pull farmos/farmos:3.x.y
docker run --rm -p 80:80 -v "${PWD}/sites:/opt/drupal/web/sites" farmos/farmos:3.x.y
Replace `3.x.y` with the desired version. Find the latest farmOS version on the
[GitHub release page](https://github.com/farmOS/farmOS/releases). Using the
`latest` Docker tag is not recommended, because updates require manual steps.
See [Updating farmOS](/hosting/update) for more info.
This will pull the farmOS Docker image, provision a farmOS web server container
listening on port 80, and bind-mount a `sites` directory into the container for
persistence of settings and uploaded files.
#### Docker Compose
[Docker Compose](https://docs.docker.com/compose) can be used to encapsulate these decisions.
An example `docker-compose.production.yml` configuration file is provided in
the farmOS repository's `docker` directory, with an accompanying `README.md`.
Copy this to a file named `docker-compose.yml` in the directory you would like
to install farmOS, update the `farmos/farmos:x.y.z` version reference, and run:
docker compose up -d
#### Persistence
All site-specific settings and user-uploaded files are stored in
`/opt/drupal/web/sites` inside the container, so it is important that the
contents of this directory be persisted outside of the container. Bind-mounting
a directory from the host into the container is the recommended way to achieve
this.
The `docker run` command above does this, as well as the example
`docker-compose.yml` provided in the farmOS repository's `docker` directory.
If the `sites` directory is not persisted, all settings and files will be lost
when the container is destroyed, and you will be prompted to install farmOS
again when a new container is started.
#### Customizing PHP
If customizations to PHP's configuration are required, such as increasing the
maximum upload size limit, you can bind-mount a custom PHP settings file into
the container.
Create a file called `php.ini` alongside `docker-compose.yml`:
```
upload_max_filesize = 50M
post_max_size = 50M
```
Bind-mount `php.ini` into the `www` service in your `docker-compose.yml` file:
```
volumes:
...
- './php.ini:/usr/local/etc/php/conf.d/farmos.ini'
```
### Packaged releases
An alternative to the Docker-based deployment is to install the farmOS codebase
directly on the host server using a packaged release tarball, available from
GitHub: [github.com/farmOS/farmOS/releases](https://github.com/farmOS/farmOS/releases)
Packaged releases include everything from the `/opt/drupal` directory in the
Docker image. This represents the entire farmOS codebase, pre-built with
[Composer](https://getcomposer.org).
Download and unpack the tarball on your web server, and point the document root
at the `web` subdirectory.
## Installing farmOS
Once you have the farmOS codebase deployed, and a database server provisioned,
you can proceed with the web-based farmOS installation. Visit the farmOS
server's hostname in your browser and follow the steps to install farmOS and
optional modules.
### File uploads
In order to upload files, a private filesystem path must be configured in the
`settings.php` file after installation is complete.
If you are using the official Docker image, and bind-mounting the `sites`
directory as a volume, add the following line to `sites/default/settings.php`:
$settings['file_private_path'] = '/opt/drupal/web/sites/default/private/files';
Additionally, create the folder `/opt/drupal/web/sites/default/private/`.
Set the correct user and permissions:
Folder ownership and group should match the web server user. If you are using
the farmOS Docker image (running Apache), this will be `www-data`.
Folder permissions should be set to `770` or `drwxrwx---`.
If you are using a packaged release outside of Docker, replace `/opt/drupal/web`
with the path to the webroot directory that contains your `sites` directory.
Finally, make sure to clear the caches by visiting Administration >
Configuration > Development > Performance and clicking the `Clear all caches`
button, or use Drush via the command line: `drush cr`.
### Cron
farmOS performs routine tasks via its built-in cron. This includes database
cleanup and garbage collection. A cron job must be configured on the host
system to trigger this. There are three ways to do this:
**1. Via the `drush cron` command (recommended)**
The Drush command-line tool provides a `cron` command for triggering the
execution of cron tasks. This can be run via a scheduled system `crontab`.
Weekly is enough for most use-cases.
The actual `drush` command will vary depending on how/where you have farmOS
installed. Drush is located in `vendor/bin/drush` within the farmOS codebase,
and may require additional arguments to find your farmOS database correctly.
For more information, see the [Drush cron documentation](https://www.drush.org/latest/cron).
If you are running farmOS in the official Docker container, your `crontab`
will look something like this (replace `[container-name]` with the name of your
running farmOS container):
`4 0 * * 0 sudo docker exec -it -u www-data [container-name] drush cron > /dev/null`
**2. Via the secret cron URL**
A special URL is available for running cron via an HTTP request. This URL
includes a secret key to prevent it from being run by unauthorized requests.
To find the secret cron URL, while logged in as a farmOS administrator, go to
Administration > Configuration > System > Cron. The URL will look like this:
`https://[base-url]/cron/[secret-key]`
Use `wget` in a Linux cron job to request this URL on a schedule. Weekly is
enough for most use-cases.
`4 0 * * 0 wget -O - -q -t 1 [cron-url] > /dev/null`
The only disadvantage of this approach is it may run into PHP `memory_limit` or
`max_execution_time` restrictions, if the cron tasks are particularly long or
complex.
**3. Via the "Automated Cron" module (not recommended)**
Drupal also includes an "Automated Cron" module, which is disabled by default
in farmOS. When enabled, this module will automatically perform cron tasks at
the end of normal visitor requests. This option is only recommended if your
host system does not have the ability to configure cron jobs. It can cause
random requests to farmOS to be very slow, as the cron tasks are tacked onto
the normal page loading process.

19
web/profiles/farm/docs/hosting/localization.md Normal file
View File

@@ -0,0 +1,19 @@
# Translating farmOS
This describes how to enable translations in farmOS.
1. Enable the **farmOS Localization** (`farm_l10n`) module.
2. Add your desired language and download translations.
1. Go to Administration > Configuration > Regional and language > Languages.
2. Click "Add language"
3. Select the desired language* and click "Add language".
4. Wait for the language translations to be downloaded and updated.
3. Go to Settings > Language and set the default language, if desired.
4. Individual users can override the default language by editing their profile.
Users that were created before changing the default language will need to be
manually updated to use the new language.
* Note that farmOS has not been fully translated into every language in the
list. To view the status of translations by language, visit
[https://localize.drupal.org/translate/projects/farm](https://localize.drupal.org/translate/projects/farm)
and select the version of farmOS that you are running.

299
web/profiles/farm/docs/hosting/migration.md Normal file
View File

@@ -0,0 +1,299 @@
# Migrating from farmOS v1
**Note: Migrating directly from farmOS v1 to v3+ is not supported. Migrate from
v1 to v2 first, then *update* to future versions using the normal
[update process](update).**
The upgrade path from farmOS v1 to v2 is performed via a database migration.
farmOS 2.x includes a **farmOS Migrate** module that leverage's Drupal core's
[Migrate API](https://drupal.org/docs/drupal-apis/migrate-api) to provide
migrations for each asset type, log type, etc. These migrations are defined in
YML configuration files included with the farmOS Migrate module.
## Important considerations
* Do not migrate into a farmOS 2.x instance that already has records. This is
to ensure that the internal auto-incrementing IDs of records are maintained.
* Execute the migrations in the *exact* order they are shown below. It is
especially important that all assets are migrated *before* any areas, because
areas are converted to assets during the migration, which can cause ID
conflicts/collisions.
See [Issue #3203228](https://www.drupal.org/project/farm/issues/3203228)
* Uploaded photos/files must be copied to the destination filesystem before
migrating. See [Migrating files](#migrating-files) below.
* If you are using the "Farm sensor: Listener" module, see
[Sensor data streams](#sensor-data-streams) below
* See [Limitations](#limitations) below.
## Running the migration
Follow the steps below to migrate your farmOS 1.x data to farmOS 2.x:
1. Install farmOS 2.x.
2. Install the farmOS modules you intend to use at `/farm/settings/modules`
(this will determine what data is migrated). If you have any community
modules installed in 1.x be sure to download and install 2.x versions.
3. Add farmOS 1.x database connection info to `settings.php`:
$databases['migrate']['default'] = [
'database' => 'my_farmos_1x_db',
'username' => 'my-db-username',
'password' => 'my-db-password',
'prefix' => '',
'host' => 'localhost',
'port' => '3306',
'namespace' => 'Drupal\\Core\\Database\\Driver\\mysql',
'driver' => 'mysql',
];
It is also recommended that you increase the PHP `memory_limit` for Drush by
adding the following to `settings.php`:
if (PHP_SAPI === 'cli') {
ini_set('memory_limit', '512M');
}
4. Copy user-uploaded files to the new directory (see
[Migrating files](#migrating-files) below).
5. Install the farmOS Migrate (`farm_migrate`) module.
6. Run the farmOS 1.x Migrations via Drush:
drush farm_migrate:import
Alternatively, migration groups can be run individually, if you need
more control over the process. They must be run in this order:
drush migrate:import --group=farm_migrate_config
drush migrate:import --group=farm_migrate_role
drush migrate:import --group=farm_migrate_user
drush migrate:import --group=farm_migrate_file
drush migrate:import --group=farm_migrate_taxonomy
drush migrate:import --group=farm_migrate_asset
drush migrate:import --group=farm_migrate_area
drush migrate:import --group=farm_migrate_asset_parent
drush migrate:import --group=farm_migrate_sensor_data
drush migrate:import --group=farm_migrate_quantity
drush migrate:import --group=farm_migrate_log
drush migrate:import --group=farm_migrate_plan
To view the status of all farmOS 1.x migrations:
drush migrate:status --tag="farmOS 1.x"
After all migrations are complete, perform a thorough examination of data to
confirm that nothing is missing or incorrect. The original 1.x database will
not be touched during the migration, so if issues are discovered it can
continue to be used as the canonical farmOS database until further testing and
debugging can be performed. See [Troubleshooting](#troubleshooting) below for
known issues.
Please open bug reports in the farmOS issue queue if new issues are discovered.
## Rolling back migration
Migrations can be rolled back with the following command:
drush farm_migrate:rollback
Alternatively, migration groups can be rolled back individually. This should be
done in the following order (reverse of the order of import):
drush migrate:rollback --group=farm_migrate_plan
drush migrate:rollback --group=farm_migrate_log
drush migrate:rollback --group=farm_migrate_quantity
drush migrate:rollback --group=farm_migrate_sensor_data
drush migrate:rollback --group=farm_migrate_asset_parent
drush migrate:rollback --group=farm_migrate_area
drush migrate:rollback --group=farm_migrate_asset
drush migrate:rollback --group=farm_migrate_taxonomy
drush migrate:rollback --group=farm_migrate_file
drush migrate:rollback --group=farm_migrate_user
drush migrate:rollback --group=farm_migrate_role
drush migrate:rollback --group=farm_migrate_config
## Migrating files
farmOS allows files to be uploaded/attached to records. In order to migrate
these files, they need to be copied into new site's files/private directories.
The farmOS migration code will look for files in the following locations:
- Public files: `public://migrate`
- Private files: `private://migrate`
The `public://` and `private://` prefixes map to the "Public file system path"
and "Private file system path" configured in farmOS 1.x and 2.x at:
`/admin/config/media/file-system`. This may vary for each installation.
For example, if you have farmOS 1.x installed in `/var/www/farmOS_1.x` and
farmOS 2.x in `/var/www/farmOS_2.x`, and both are configured to use
`sites/default/files` for public files, and `sites/default/private` for private
files, then copy the files as follows:
cp -rp /var/www/farmOS_1.x/sites/default/files /var/www/farmOS_2.x/web/sites/default/files/migrate
cp -rp /var/www/farmOS_1.x/sites/default/private/files /var/www/farmOS_2.x/web/sites/default/private/files/migrate
The farmOS migration code will automatically move files from `files/migrate/*`
to `files/*`. Only the files that it finds in the `{file_managed}` table will
be moved, leaving behind various temporary files in the `migrate` directory
that are no longer needed after the migration. This `migrate` directory can be
deleted after the migration, once it has been confirmed that everything was
migrated successfully.
## Sensor data streams
If you are using the "Farm sensor: Listener" module in farmOS 1.x to collect
data from sensors, there are a few extra steps and considerations for migrating
and maintaining these data streams in farmOS 2.x.
farmOS 2.x introduces a new entity type called "Data streams", which represent
named sets of time-series data. All data from the old "listener" module can be
migrated into named data streams.
In order to migrate data, you must enable the "Sensor listener (legacy)" module
in farmOS 2.x. This can be found in Drupal core's `/admin/modules` page, or it
can be enabled via Drush:
drush en farm_sensor_listener
Enabling this module does two things:
1. Adds a migration for 1.x sensor data into 2.x data streams.
2. Creates URL endpoints that match the legacy listener paths, which ensures
that farmOS will continue receiving and storing data being pushed from your
sensors.
If you have active sensors that are sending data to your farmOS, and you want
to minimize downtime/interruption of these streams during migration, one
approach is to set up your 2.x instance on a different domain, run migrations,
and then point your domain to the new server. Be sure to set the TTL value of
your domain's DNS record to 5 minutes or less to ensure that the change
propagates quickly. It is highly recommended that you test the migrations at
least once before this, to ensure that they all work smoothly.
If you do not have any active sensors sending data, then you can optionally
uninstall the `farm_sensor_listener` module after running migrations. This will
remove the legacy API endpoints, but still keep your migrated data.
## Limitations
The farmOS migration code is designed to migrate a *default* farmOS 1.x
database to 2.x. If any customizations have been made on top of the defaults,
they will not be migrated.
This includes (but is not limited to):
- Custom asset, entity, taxonomies, and log types
- Custom fields
- Custom roles
If you maintain a contrib/custom module for farmOS 1.x, it is your
responsibility to update the modules for 2.x and provide migration logic.
## Troubleshooting
### Validation
Validation is performed on all areas, assets, logs, plans, and taxonomy terms
as they are migrated. This will check things like required fields, allowed
values, etc. In some cases the data in a 1.x database will not pass validation,
either because it was not properly validated originally, or due to legacy bugs
in the farmOS 1.x code. If any entities fail validation, migration will stop
and an error like the following will be displayed:
farm_migrate_asset_plant Migration - 1 failed.
You can view validation messages for individual migrations by running
`drush migrate:messages [migration-id]`, which will provide more details. For
example:
$ drush migrate:messages farm_migrate_asset_plant
-------------- ------------------- ------- ---------------------------------------------------------
Source ID(s) Destination ID(s) Level Message
-------------- ------------------- ------- ---------------------------------------------------------
432 1 [asset: 432]: plant_type=This value should not be null.
-------------- ------------------- ------- ---------------------------------------------------------
This gives you the opportunity to fix the data in your 1.x database. Then you
can rollback and re-run the migration, like so:
drush migrate:rollback farm_migrate_asset_plant
drush migrate:import farm_migrate_asset_plant
### Reset status
If an error occurs during migration, the status of the broken migration may be
stuck as "Importing". In order to rerun the migration, first reset the status
and then roll back the migration. Replace `[migration_id]` with ID of the
migration that is stuck.
drush migrate:reset-status [migration_id]
drush migrate:rollback [migration_id]
### Memory limit
If you have a large amount of data, there is a chance you may encounter run
into the PHP memory limit. You will see an error like the following:
Fatal error: Allowed memory size of 268435456 bytes exhausted
If this happens, you can increase the PHP `memory_limit` setting for Drush by
tweaking the `ini_set('memory_limit', '512M');` line in your `settings.php`,
assuming your host's memory can accommodate it.
**Be sure to roll back any migrations that were being processed when the error
occurred to ensure that no data is corrupted.**
### Movement logs
farmOS 2.x changes the way asset movements are described via logs. There is a
single "Location reference" and "Geometry" field on logs now, as opposed to
the separate "Move to" and "Movement geometry" fields that existed in 1.x. The
migration will use the movement area references and geometry if they are
present, and will automatically mark the log as a movement.
However, if the log has additional area references and geometry data, then the
migration logic will detect the conflict and one of the following errors will
be thrown:
> Log 123 has both area references and movement area references.
> Log 123 has both a geometry and a movement geometry.
If these errors are encountered, the migration will halt and can not be
completed until either:
1. the logs in the old database are cleaned up, or
2. the migration script is explicitly allowed to overwrite non-movement area
references and geometry
Manual clean up involves reviewing the logs that cause errors in the old
database, deleting the "Areas" and "Geometry" fields (or copying them into the
"Move to" and "Movement geometry" fields), and retrying the migration. In some
cases it may make sense to split the log into two separate logs, in order to
retain information.
Alternatively, the migration script can be allowed to automatically overwrite
the "Areas" and "Geometry" data from the log, and only keep the "Move to" and
"Movement geometry" data. This can be configured by adding the following line
to `settings.php`:
$settings['farm_migrate_allow_movement_overwrite'] = TRUE;
**Beware that this may result in loss of data/context if the separate fields
were being used intentionally. It is recommended that logs be reviewed manually to
understand whether or not the data is needed.**
After running the migration with this setting, warnings for each log will be
stored, and can be viewed with:
drush migrate:messages [migration_id]
### Quantities
The farmOS 2.x migration creates all Quantity entities before it creates the
Log entities that reference them. This means that it is possible to end up with
orphaned quantities, if for instance you do not migrate all of your log types
from farmOS 1.x. There is no built-in way to clean these up currently, so it is
recommended that all log types be migrated.

98
web/profiles/farm/docs/hosting/update.md Normal file
View File

@@ -0,0 +1,98 @@
# Updating farmOS
**ALWAYS BACKUP YOUR DATABASE, CODE, AND FILES BEFORE ATTEMPTING AN UPDATE!**
New versions of farmOS are released on a regular basis, and it's important to
stay up-to-date so that you can receive new features, bug fixes, and security
patches when they become available. Find the latest farmOS version on the
[GitHub release page](https://github.com/farmOS/farmOS/releases).
## Update procedure
1. **Backup your database and files!** Always do this before updating. Be ready
and able to roll-back in the event that something goes wrong. Typically this
can be done by creating a database dump and a tarball of your `web/sites`
directory. If you are using Docker, be sure you can roll back to the
previous image version. If you are using packaged releases, be sure you
also keep a copy of the old farmOS codebase before updating.
2. **Update the farmOS codebase.** This will depend on how you have deployed
farmOS. See [Updating via Docker](#updating-via-docker) or
[Updating via packaged releases](#updating-via-packaged-releases) below for
specific instructions. If you are building a custom farmOS codebase with
Composer see
[Updating dependencies](/hosting/composer#updating-dependencies).
3. **Run automated updates.** Navigate to `https://[hostname]/update.php` in
your browser and follow the steps to run automated updates. It is important
to do this before using the new version of farmOS to ensure that any
necessary changes to the database or configuration are made.
4. **Clear caches.** farmOS caches can be cleared by going to
`https://[hostname]/admin/config/development/performance` in your browser
and clicking "Clear all caches", or via the command line with Drush:
`drush cr`. Cache clearing is only necessary if no updates are performed
during `update.php`, otherwise they will be cleared automatically.
### Updating via Docker
If you have [deployed farmOS via Docker](/hosting/install#farmos-in-docker) you
can update to a new version of farmOS by pulling the new Docker image version:
docker pull farmos/farmos:[version]
This can also be done by updating your `docker-compose.yml` file if you are
using Docker Compose and restarting your containers.
Assuming that `/opt/drupal/web/sites` is the only directory persisted outside
of the container, this will update the farmOS codebase, which is located in
`/opt/drupal` within the container.
See [Persistence](/hosting/install#persistence) for more information.
### Updating via packaged releases
If you have [deployed farmOS via packaged releases](/hosting/install#packaged-releases)
you can update to a new version of farmOS by downloading the new release
tarball and unpacking it in place of the old codebase, replacing everything
except the `web/sites` directory. Be sure to *replace* (not *merge*) all files
and directories.
**Do not overwrite the `web/sites` directory, because it contains all your
site-specific settings and uploaded files.**
### Maintenance mode
Optionally, you may put farmOS into "maintenance mode" to prevent users from
accessing it during the update process.
Navigate to `https://[hostname]/admin/config/development/maintenance` in your
browser, enable maintenance mode, perform the update, confirm the new version
works, then disable maintenance mode. If you are the only user of farmOS then
this is not necessary, but be sure to run `update.php` before using the new
version.
### Drush
An alternative to running the automated updates by visiting `update.php` in
your browser is to use the command-line tool [Drush](https://www.drush.org).
The Drush command for running updates is:
drush updb
If you are running farmOS in Docker with the standard Docker Compose
configuration, you can run this command inside the container with:
docker compose exec www drush updb
## Security releases
When there are security updates released for farmOS dependency modules, you may
see the following message:
> There is a security update available for your version of Drupal. To ensure
> the security of your server, you should update immediately! See the available
> updates page for more information.
The farmOS maintainers will update these dependencies in the development branch
of farmOS as soon as possible, and if the vulnerabilities affect farmOS
directly a new version will be tagged and released. Many vulnerabilities are
mitigated by the fact that farmOS is only accessible to users with a login.
In those cases the maintainers may deem a release unnecessary. This message can
be disabled by uninstalling the "Update Manager" module.