Update tt configuration and env description to 2.0 (#4033)

Resolves #3997 
Resolves #3998 
Resolves #3836

Author: p7nov

doc/book/admin/logs.rst

@@ -7,7 +7,7 @@ Each Tarantool instance logs important events to its own log file ``<instance-na
 For instances started with :ref:`tt <tt-cli>`, the log location is defined by
 the ``log_dir`` parameter in the :ref:`tt configuration <tt-config>`.
 By default, it's ``/var/log/tarantool`` in the ``tt`` :ref:`system mode <tt-config_modes>`,
-and the ``var/log/`` subdirectory of the ``tt`` working directory in the :ref:`local mode <tt-config_modes>`.
+and the ``var/log`` subdirectory of the ``tt`` working directory in the :ref:`local mode <tt-config_modes>`.
 In the specified location, ``tt`` creates separate directories for each instance's logs.
 
 To check how logging works, write something to the log using the :ref:`log <log-module>` module:
@@ -41,31 +41,35 @@ Then check the logs:
 Log rotation
 ------------
 
-When :ref:`logging to a file <cfg_logging-log>`, the system administrator must ensure logs are
-rotated timely and do not take up all the available disk space.
-To prevent log files from growing infinitely, ``tt`` automatically rotates instance
-logs. The following ``tt`` configuration parameters define the log rotation:
-``log_maxsize`` (in megabytes) and ``log_maxage`` (in days). When any of these
-limits is reached, the log is rotated.
-Additionally, there is the ``log_maxbackups`` parameter (the number of stored log
-files for an instance), which enables automatic removal of old log files.
-
-..  code-block:: yaml
-
-    # tt.yaml
-    tt:
-      app:
-        log_maxsize: 100
-        log_maxage: 3
-        log_maxbackups: 50
-        # ...
-
-There is also the :ref:`tt logrotate <tt-logrotate>` command that performs log
-rotation on demand.
-
-..  code-block:: bash
-
-    tt logrotate my_app
+When :ref:`logging to a file <cfg_logging-log>`, the system administrator must ensure
+logs are rotated timely and do not take up all the available disk space.
+The recommended way to prevent log files from growing infinitely is using an external
+log rotation program, for example, ``logrotate``, which is pre-installed on most
+mainstream Linux distributions.
+
+A Tarantool log rotation configuration for ``logrotate`` can look like this:
+
+.. code-block:: text
+
+    /var/log/tarantool/*.log {
+        daily
+        size 512k
+        missingok
+        rotate 10
+        compress
+        delaycompress
+        create 0640 tarantool adm
+        postrotate
+            tt -c <tt_config_file> logrotate
+        endscript
+    }
+
+In this configuration, :ref:`tt logrotate <tt-logrotate>` is called after each log
+rotation to reopen the instance log files after they are moved by the ``logrotate``
+program.
+
+There is also the built-in function :ref:`log.rotate() <log-rotate>`, which you
+can call on an instance to reopen its log file after rotation.
 
 To learn about log rotation in the deprecated ``tarantoolctl`` utility,
 check its :ref:`documentation <tarantoolctl-log-rotation>`.

doc/reference/tooling/tt_cli/configuration.rst

@@ -14,36 +14,63 @@ execution is its configuration file.
 By default, the configuration file is called ``tt.yaml``. The location
 where ``tt`` searches for it depends on the :ref:`launch mode <tt-config_modes>`.
 You can also pass the configuration file explicitly in the ``--cfg``
-:ref:`option <tt-global-options>`.
+:ref:`global option <tt-global-options>`.
 
-The ``tt`` configuration file is a YAML file with the following content:
+The ``tt`` configuration file is a YAML file with the following structure:
 
 ..  code-block:: yaml
 
-    tt:
-      modules:
-        directory: path/to/modules/dir
-      app:
-        instances_enabled: path/to/applications
-        run_dir: path/to/run_dir
-        log_dir: path/to/log_dir
-        bin_dir: path/to/bin_dir
-        inc_dir: path/to/inc_dir
-        wal_dir: path/to/wal_dir
-        vinyl_dir: path/to/vinyl_dir
-        memtx_dir: path/to/memtx_dir
-        log_maxsize: num (MB)
-        log_maxage: num (days)
-        log_maxbackups: num
-        restart_on_failure: bool
-      repo:
-        rocks: path/to/rocks
-        distfiles: path/to/install
-      ee:
-        credential_path: path/to/file
-      templates:
-        - path: path/to/app/templates1
-        - path: path/to/app/templates2
+    env:
+      instances_enabled: path/to/available/applications
+      bin_dir: path/to/bin_dir
+      inc_dir: path/to/inc_dir
+      restart_on_failure: bool
+      tarantoolctl_layout: bool
+    modules:
+      directory: path/to/modules/dir
+    app:
+      run_dir: path/to/run_dir
+      log_dir: path/to/log_dir
+      wal_dir: path/to/wal_dir
+      vinyl_dir: path/to/vinyl_dir
+      memtx_dir: path/to/memtx_dir
+    repo:
+      rocks: path/to/rocks
+      distfiles: path/to/install
+    ee:
+      credential_path: path/to/file
+    templates:
+      - path: path/to/app/templates1
+      - path: path/to/app/templates2
+
+.. note::
+
+    The ``tt`` configuration format and application layout have been changed in version
+    2.0. Learn how to upgrade from earlier versions in :ref:`tt-config_migrating-from-1`.
+
+.. _tt-config_file_env:
+
+env section
+~~~~~~~~~~~
+
+.. note::
+
+    The paths specified in ``env.*`` parameters are relative to the current ``tt``
+    environment's root.
+
+*   ``instances_enabled`` -- the directory where :ref:`instances <admin-instance_file>`
+    are stored. Default: ``instances.enabled``.
+*   ``bin_dir`` -- the directory where binary files are stored. Default: ``bin``.
+*   ``inc_dir`` -- the base directory for storing header files. They will
+    be placed in the ``include`` subdirectory inside the specified directory.
+    Default: ``include``.
+*   ``restart_on_failure`` -- restart the instance on failure: ``true`` or ``false``.
+    Default: ``false``.
+*   ``tarantoolctl_layout`` -- use a layout compatible with the deprecated ``tarantoolctl``
+    utility for artifact files: control sockets, ``.pid`` files, log files.
+    Default: ``false``.
+
+.. _tt-config_file_modules:
 
 modules section
 ~~~~~~~~~~~~~~~
@@ -56,44 +83,25 @@ modules section
 app section
 ~~~~~~~~~~~
 
-*   ``instances_enabled`` -- the directory where :ref:`instances <admin-instance_file>`
-    are stored.
+.. note::
+
+    The paths specified in ``app.*_dir`` parameters are relative to the application
+    location inside the ``instances.enabled`` directory specified in the ``env``
+    configuration section. For example, the default location of the ``myapp``
+    application's logs is ``instances.enabled/myapp/var/log``.
+    Inside this location, ``tt`` creates separate directories for each application
+    instance that runs in the current environment.
+
 *   ``run_dir``-- the directory for instance runtime artifacts, such as console
     sockets or PID files. Default: ``var/run``.
 *   ``log_dir`` -- the directory where log files are stored. Default: ``var/log``.
-*   ``bin_dir`` -- the directory where binary files are stored. Default: ``bin``.
-*   ``inc_dir`` -- the base directory for storing header files. They will
-    be placed in the ``include`` subdirectory inside the specified directory.
-    Default: ``include``.
 *   ``wal_dir`` -- the directory where write-ahead log (``.xlog``) files are stored.
     Default: ``var/lib``.
 *   ``memtx_dir`` -- the directory where memtx stores snapshot (``.snap``) files.
     Default: ``var/lib``.
 *   ``vinyl_dir`` -- the directory where vinyl files or subdirectories are stored.
     Default: ``var/lib``.
 
-    .. note::
-
-        In all directories specified in ``*_dir`` parameters, ``tt`` creates a
-        directory for each application and instance directories inside it.
-        Names of these directories match the names of applications and instances.
-
-*   ``log_maxsize`` -- the maximum size of the log file before it gets rotated,
-    in megabytes. Default: 100.
-*   ``log_maxage`` -- the maximum age of log files in days. The age of a log
-    file is defined by the timestamp encoded in its name. Default: not defined
-    (log files aren't deleted based on their age).
-
-    ..  note::
-
-        A day is defined as exactly 24 hours. It may not exactly correspond to
-        calendar days due to daylight savings, leap seconds, and other time adjustments.
-
-*   ``log_maxbackups`` -- the maximum number of stored log files.
-    Default: not defined (log files aren't deleted based on their count).
-*   ``restart_on_failure`` -- restart the instance on failure: ``true`` or ``false``.
-    Default: ``false``.
-
 .. _tt-config_file_repo:
 
 repo section
@@ -131,7 +139,7 @@ configuration file. There are three launch modes:
 Default launch
 ~~~~~~~~~~~~~~
 
-**Argument**: none
+**Global option**: none
 
 **Configuration file**: searched from the current directory to the root.
 Taken from ``/etc/tarantool`` if the file is not found.
@@ -143,7 +151,7 @@ Taken from ``/etc/tarantool`` if the file is not found.
 System launch
 ~~~~~~~~~~~~~
 
-**Argument**: ``--system`` or ``-S``
+**Global option**: ``--system`` or ``-S``
 
 **Configuration file**: Taken from ``/etc/tarantool``.
 
@@ -154,10 +162,68 @@ System launch
 Local launch
 ~~~~~~~~~~~~
 
-**Argument**: ``--local=DIRECTORY`` or ``-L=DIRECTORY``
+**Global option**: ``--local=DIRECTORY`` or ``-L=DIRECTORY``
 
 **Configuration file**: Searched from the specified directory to the root.
 Taken from ``/etc/tarantool`` if the file is not found.
 
 **Working directory**: The specified directory. If ``tarantool`` or ``tt``
-executable files are found in the working directory, they will be used.
+executable files are found in the working directory, they will be used.
+
+.. _tt-config_migrating-from-1:
+
+Migrating from tt 1.* to 2.0 or later
+-------------------------------------
+
+The `tt` configuration and application layout were changed in version 2.0.
+If you are using ``tt`` 1.*, complete the following steps to migrate to ``tt`` 2.0 or later:
+
+#.  **Update the tt configuration file**.
+    In tt 2.0, the following changes were made to the configuration file:
+
+    *   The root section ``tt`` was removed. Its child sections -- ``app``, ``repo``,
+        ``modules``, and other -- have been moved to the top level.
+    *   Environment configuration parameters were moved from the ``app`` section
+        to the new section ``env``. These parameters are ``instances.enabled``,
+        ``bin_dir``, ``inc_dir``, and ``restart_on_failure``.
+    *   The paths in the ``app`` section are now relative to the app directory in ``instances.enabled``
+        instead of the environment root.
+
+    You can use :ref:`tt init <tt-init>` to generate a configuration file with
+    the new structure and default parameter values.
+
+#.  **Move application artifacts**.
+    With ``tt`` 1.*, application artifacts (logs, snapshots, pid, and other files)
+    were created in the ``var`` directory inside the *environment root*. Starting from
+    ``tt`` 2.0, these artifacts are created in the ``var`` directory inside the
+    *application directory*, which is ``instances.enabled/<app-name>``. This is
+    how an application directory looks:
+
+    .. code-block:: text
+
+        instances.enabled/app/
+        ├── init.lua
+        ├── instances.yml
+        └── var
+            ├── lib
+            │   ├── instance1
+            │   └── instance2
+            ├── log
+            │   ├── instance1
+            │   └── instance2
+            └── run
+                ├── instance1
+                └── instance2
+
+    To continue using existing application artifacts after migration from ``tt`` 1.*:
+
+    #.  Create the ``var`` directory inside the application directory.
+    #.  Create the ``lib``, ``log``, and ``run`` directories inside ``var``.
+    #.  Move directories with instance artifacts from the old ``var`` directory
+        to the new ``var`` directories in applications' directories.
+
+#.  **Move the files accessed from the application code**.
+    The working directory of instance processes was changed from the ``tt`` working
+    directory to the application directory inside ``instances.enabled``. If the
+    application accesses files using relative paths, move the files accordingly
+    or adjust the application code.

doc/reference/tooling/tt_cli/global_options.rst

@@ -3,9 +3,18 @@
 Global options
 ==============
 
+.. important::
+
+    Global options of ``tt`` must be passed before its commands and other options.
+    For example:
+
+    .. code-block:: console
+
+        $ tt --cfg tt-conf.yaml start app
+
 ``tt`` has the following global options:
 
-.. option:: -c=FILE, --cfg=FILE
+.. option:: -c=FILE, --cfg=FILE,
 
     Path to the :ref:`configuration file <tt-config_file>`.
 

doc/reference/tooling/tt_cli/logrotate.rst

@@ -7,13 +7,18 @@ Rotating instance logs
 
     $ tt logrotate {INSTANCE | APPLICATION[:APP_INSTANCE]}
 
-``tt logrotate`` rotates logs of a specified Tarantool instance.
+``tt logrotate`` rotates logs of a Tarantool application or specific instances,
+and the ``tt`` log. For example, you need to call this function to continue logging
+after a log rotation program renames or moves instances' logs.
 Learn more about :ref:`rotating logs <admin-logs>`.
 
+Calling ``tt logrotate`` on an application has the same effect as executing the
+built-in :ref:`log.rotate() <log-rotate>` function on all its instances.
+
 Examples
 --------
 
-Rotate logs of the ``app`` instance:
+Rotate logs of the ``app`` application's instances:
 
 ..  code-block:: console