ethereum-optimism · cpengilly · May 15, 2024 · May 9, 2024 · May 9, 2024 · May 13, 2024
@@ -161,5 +161,5 @@ If you do it this way, you won't have to wait until the transactions are written
 
 ## Next Steps
 
-*   See the [Node Configuration](/builders/node-operators/management/configuration#configuration) guide for additional explanation or customization.
+*   See the [Node Configuration](/builders/node-operators/configuration/base-config) guide for additional explanation or customization.
 *   If you experience difficulty at any stage of this process, please reach out to [developer support](https://github.com/ethereum-optimism/developers/discussions).
@@ -23,16 +23,16 @@ To enable snap sync, chain operators need to spin up a node which is exposed to
 
 <Callout type="info">
 For snap sync, all `op-geth` nodes should expose port `30303` TCP and `30303` UDP to easily find other op-geth nodes to sync from. 
-  *   If you set the port with [`--discovery.port`](/builders/node-operators/management/configuration#discoveryport), then you must open the port specified for UDP. 
-  *   If you set [`--port`](/builders/node-operators/management/configuration#port), then you must open the port specified for TCP.
+  *   If you set the port with [`--discovery.port`](/builders/node-operators/configuration/execution-config#discoveryport), then you must open the port specified for UDP. 
+  *   If you set [`--port`](/builders/node-operators/configuration/execution-config#port), then you must open the port specified for TCP.
   *   The only exception is for sequencers and transaction ingress nodes.
 </Callout>
 
 <Steps>
   ### Setup a Snap Sync Node
 
   *   Expose port `30303` (`op-geth`'s default discovery port) to the internet on TCP and UDP.
-  *   Disable transaction gossip with the [`--rollup.disabletxpoolgossip`](/builders/node-operators/management/configuration#rollupdisabletxpoolgossip) flag
+  *   Disable transaction gossip with the [`--rollup.disabletxpoolgossip`](/builders/node-operators/configuration/execution-config#rollupdisabletxpoolgossip) flag
 
   ### Enable Snap Sync on Your Network
 
@@ -41,5 +41,5 @@ For snap sync, all `op-geth` nodes should expose port `30303` TCP and `30303` UD
 
 ## Next Steps
 
-*   See the [Node Configuration](/builders/node-operators/management/configuration#configuration) guide for additional explanation or customization.
+*   See the [Node Configuration](/builders/node-operators/configuration/base-config#configuration) guide for additional explanation or customization.
 *   If you experience difficulty at any stage of this process, please reach out to [developer support](https://github.com/ethereum-optimism/developers/discussions).
@@ -677,7 +677,7 @@ You can add the following options to the `op-node` command to enable peer-to-pee
   --p2p.listen.udp=9003 \
 ```
 
-You can alternatively also remove the [--p2p.static](/builders/node-operators/management/configuration#p2pstatic) option, but you may see failed requests from other chains using the same chain ID.
+You can alternatively also remove the [--p2p.static](/builders/node-operators/configuration/consensus-config#p2pstatic) option, but you may see failed requests from other chains using the same chain ID.
 </Callout>
 
 </Steps>

@@ -1,6 +1,8 @@
 {
   "overview": "Overview",
+  "architecture": "Architecture",
   "tutorials": "Tutorials",
+  "configuration": "Configuration",
   "management": "Node Management",
   "network-upgrades": "Network Upgrades",
   "json-rpc": "JSON-RPC API",

@@ -0,0 +1,42 @@
+---
+title: Node Architecture
+lang: en-US
+description: Learn about node architecture.
+---
+
+# Node Architecture
+
+This page reviews node architecture for all nodes running on the Superchain network. All OP Mainnet nodes are composed of two core software services, the Rollup Node and the Execution Client.
+OP Mainnet also optionally supports a third component, Legacy Geth, that can serve stateful queries for blocks and transactions created before the [Bedrock Upgrade](https://web.archive.org/web/20230608050602/https://blog.oplabs.co/introducing-optimism-bedrock/).
+
+## Node Flow Diagram
+The following diagram shows how the Rollup Node, Execution Client, and Legacy Geth components work together to form a complete node running on the Superchain network.
+This diagram uses the `op-node` and `op-geth` implementations of the Rollup Node and Execution Client respectively, but the same architecture generally applies to other implementations as well.
+
+![OP Mainnet node architecture diagram.](/img/guides/node-operators/node-arch.svg)
+
+## Rollup Node
+
+The Rollup Node is responsible for deriving L2 block payloads from L1 data and passing those payloads to the Execution Client. The Rollup Node can also optionally participate in a peer-to-peer network to receive blocks directly from the Sequencer before those blocks are submitted to L1. The Rollup Node is largely analogous to a [consensus client](https://ethereum.org/en/developers/docs/nodes-and-clients/#what-are-nodes-and-clients) in Ethereum.
+
+## Execution Client
+
+The Execution Client is responsible for executing the block payloads it receives from the Rollup Node over JSON-RPC via the standard [Ethereum Engine API](https://github.com/ethereum/execution-apis/blob/main/src/engine/common.md#engine-api----common-definitions).
+The Execution Client exposes the standard JSON-RPC API that Ethereum developers are familiar with, and can be used to query blockchain data and submit transactions to the network.
+The Execution Client is largely analogous to an [execution client](https://ethereum.org/en/developers/docs/nodes-and-clients/#what-are-nodes-and-clients) in Ethereum.
+
+## Legacy Geth
+
+OP Mainnet underwent a large database migration as part of the [Bedrock Upgrade](https://web.archive.org/web/20230608050602/https://blog.oplabs.co/introducing-optimism-bedrock/) in 2023.
+Blocks and transactions included in OP Mainnet prior to the Bedrock Upgrade are served by current execution engines but cannot be executed without the help of a special component called Legacy Geth.
+This means that you will need to run Legacy Geth if you want to be able to run RPC calls such as `eth_call` on blocks before the Bedrock Upgrade.
+
+Legacy Geth is the software that was used to run OP Mainnet nodes prior to the Bedrock Upgrade.
+If you run an instance of Legacy Geth alongside your OP Mainnet node, your node will be able to forward requests against historical transactions to the Legacy Geth instance.
+Legacy Geth is **not** required and is typically only necessary if you want to maintain a complete archive node for OP Mainnet.
+
+## Next Steps
+
+*   To get your node up and running, start with the [run a node from docker](/builders/node-operators/tutorials/node-from-docker) or [build a node from source](/builders/node-operators/tutorials/node-from-source) tutorial.
+*   If you've already got your node up and running, check out the [Node Metrics and Monitoring Guide](./management/metrics) to learn how to keep tabs on your node and make sure it keeps running smoothly.
+*   If you run into any problems, please visit the [Node Troubleshooting Guide](./management/troubleshooting) for help.
@@ -0,0 +1,5 @@
+{
+    "base-config": "Base Configuration",
+    "consensus-config": "Consensus Layer Config",
+    "execution-config": "Exeuction Layer Config"
+  }
@@ -0,0 +1,221 @@
+---
+title: Node Base Configuration
+lang: en-US
+description: Learn the node base configuration and recommended flags for op-node, op-geth, and legacy geth.
+---
+
+import { Callout } from 'nextra/components'
+
+# Node Base Configuration
+
+<Callout type="warning">
+  Always run `op-node` and `op-geth` in a one-to-one configuration. Don't run multiple `op-geth` instances behind one `op-node`, or vice versa.
+</Callout>
+
+To configure your node, you will need to do the following:
+
+1.  Configure `op-node` to point to the correct L1, `op-geth`, and L2 network.
+2.  Initialize `op-geth` with the correct network parameters.
+3.  Configure `op-geth` to properly communicate with the Rollup Node.
+4.  Optionally, configure Legacy Geth.
+
+## Configuring `op-geth`
+
+<Callout type="info">
+  Although the Docker image for the Execution Engine is called `op-geth`, the actual binary is still called `geth` in order to minimize differences between `op-geth` and `go-ethereum`. You can see the difference [here](https://op-geth.optimism.io/).
+</Callout>
+
+`op-geth` stores its state in a database that requires initialization. 
+Depending on the network you're running, initialization is done one of three ways:
+
+1.  **With Network Flags:** This initializes the genesis information and chain configuration from the [superchain-registry](https://github.com/ethereum-optimism/superchain-registry).
+2.  **With a Genesis File:** This is for deployments that are not migrated from a legacy network (i.e. OP Sepolia). In this case, you'll use a genesis file and initialize the data directory via `geth init`.
+3.  **With a Data Directory:** This is used for networks that are migrated from a legacy network. This currently **only** includes OP Mainnet. In this case, you'll download a preconfigured data directory and extract it. No further initialization is necessary in this case, because the data directory contains the network's genesis information. This method can be bypassed if you utilize [snap sync](/builders/node-operators/management/snap-sync).
+
+Regardless of how `op-geth` is initialized, you'll need to ensure that you have sufficient disk space available to store the network's data. As of this writing, the OP Mainnet data directory is \~1.6TB for a full node and \~5TB for an archival node.
+
+### Initialize `op-geth`
+Instructions for each initialization method are below. If you're spinning up an OP Mainnet, use the [Initialization via Data Directory](#initialization-via-data-directory) path. If you're spinning up an OP Sepolia node, use the [Initialization via Network Flags](#initialization-via-network-flags) path.
+
+#### Initialization via Network Flags
+
+To initialize `op-geth` with the network flags, you simply need to set the 
+`--op-network=<value>` and `--network=<value>` on `op-node`. To see the latest
+support networks, you can consult the `--help` output for the `op-network` 
+option.
+
+#### Initialization via Genesis File
+
+`op-geth` uses JSON files to encode a network's genesis information. For 
+networks that are initialized in this way, you'll receive a URL to the genesis 
+JSON. You'll need to download the genesis JSON, then run the following command 
+to initialize the data directory:
+
+```bash
+#!/bin/sh
+FILE=/$DATADIR/genesis.json
+OP_GETH_GENESIS_URL=<<insert op-geth url to the genesis file>>
+
+if [ ! -s $FILE ]; then
+  apk add curl
+  curl $OP_GETH_GENESIS_URL -o $FILE
+  geth init --datadir /db $FILE
+else
+  echo "Genesis file already exists. Skipping initialization."
+fi
+```
+
+#### Initialization via Data Directory
+
+To initialize `op-geth` with a preconfigured data directory, simply download and extract the data directory to a place of your choosing. The data directory is exported as a tar file. An example command to do this is below:
+
+```bash
+curl -o <path to data directory> -sL <URL to data directory>
+tar -xvf <path to data directory>
+```
+
+### Configuration
+
+Once `op-geth` is initialized, it can be configured via CLI flags. `op-geth` accepts all the [standard `go-ethereum` flags](https://geth.ethereum.org/docs/interface/command-line-options) as well as a few extra flags that are specific to Optimism. These flags are:
+
+*   `--rollup.historicalrpc`: Enables the historical RPC endpoint. This endpoint is used to fetch historical execution data from Legacy Geth. This flag is only necessary for upgraded networks.
+*   `--rollup.sequencerhttp`: HTTP endpoint of the sequencer. `op-geth` will route `eth_sendRawTransaction` calls to this URL. Bedrock does not currently have a public mempool, so this is required if you want your node to support transaction submission. Consult the documentation for the network you are participating in to get the correct URL.
+*   `--rollup.disabletxpoolgossip`: Disables transaction pool gossiping. While not required, it's useful to set this to `true` since transaction pool gossip is currently unsupported.
+
+To communicate with `op-node` and enable the Engine API, you'll also need to generate a JWT secret file and enable Geth's authenticated RPC endpoint.
+
+To generate the JWT secret, run the following:
+
+```bash
+openssl rand -hex 32 > jwt.txt
+```
+
+Then, specify the following flags:
+
+*   `--authrpc.addr`: Sets the address `op-geth`'s authenticated RPC should listen on.
+*   `--authrpc.port`: Sets the port `op-geth`'s authenticated RPC should listen on. The default value is `8551`.
+*   `--authrpc.jwtsecret`: Sets the path to a JWT secret file you generated above.
+
+### Recommended Flags for `op-geth` Configuration
+
+You may also want to specify the following flags based on your configuration:
+
+*   `--authrpc.vhosts`: Whitelists which hosts (as defined in the `Host` header) are allowed to access the authenticated RPC endpoint. This is useful if you're running `op-geth` on containerized infrastructure. The default value is `localhost`.
+*   `--http.vhosts`: Whitelists which hosts (as defined in the `Host` header) are allowed to access the unauthenticated RPC endpoint. This is useful if you're running `op-geth` on containerized infrastructure. The default value is `localhost`.
+*   `--http`, `--http.addr`, and `--http.port`: Enables the unauthenticated RPC endpoint, configures its address, and configures its port. You'll almost certainly want to specify these, since they will enable Geth's JSON-RPC endpoint.
+*   `--ws`, `--ws.addr`, and `--ws.port`: Enables the WebSocket API.
+*   `--verbosity`: Configures Geth's log level. This is a number between 0 and 5, with 5 being the most verbose. Defaults to 3.
+
+### Working Base Configuration
+
+A valid command that runs `op-geth` and enables RPC over HTTP and WebSockets looks like:
+
+```bash
+geth \
+  --ws \
+  --ws.port=8546 \
+  --ws.addr=localhost \
+  --ws.origins="*" \
+  --http \
+  --http.port=8545 \
+  --http.addr=localhost \
+  --http.vhosts="*" \
+  --http.corsdomain="*" \
+  --authrpc.addr=localhost \
+  --authrpc.jwtsecret=/var/secrets/jwt.txt \
+  --authrpc.port=8551 \
+  --authrpc.vhosts="*" \
+  --datadir=/data \
+  --verbosity=3 \
+  --rollup.disabletxpoolgossip=true \
+  --rollup.sequencerhttp=https://mainnet-sequencer.optimism.io/ \
+  --op-network=op-mainnet
+```
+
+Consult [Geth's documentation](https://geth.ethereum.org/docs/) for more information on customizing `op-geth`'s behavior.
+
+## Configuring `op-node`
+
+`op-node` is a standalone, statically linked binary. It stores no state, and requires no initialization. It consumes configuration parameters either via the command line or environment variables. For some networks, the Rollup Node also requires a configuration file (called `rollup.json` or the "rollup config") that configures network-specific genesis parameters. For official networks like OP Sepolia and OP Mainnet, the genesis config is hardcoded in the `op-node` software and can be specified via a `--network` flag.
+
+Following the [Ecotone upgrade](/builders/node-operators/network-upgrades#ecotone)
+node operators must set an L1 beacon value to retrieve [blobs](/builders/node-operators/management/blobs)
+from a Beacon node.
+
+<Callout type="warning">
+  The `op-node` RPC should not be exposed publicly. If left exposed, it could
+  accidentally expose admin controls to the public internet.
+</Callout>
+
+### Working Base Configuration
+
+A minimal valid configuration that runs `op-node` looks like:
+
+```bash
+op-node --l1=<ethereum mainnet RPC url> \
+        --l2=<op-geth authenticated RPC url> \
+        --network=op-mainnet \
+        --rpc.addr=127.0.0.1 \
+        --rpc.port=9545 \
+        --l2.jwt-secret=<path to JWT secret> \
+        --l1.beacon=<http endpoint address of L1 Beacon-node> \
+        --syncmode=execution-layer
+```
+
+You can manually specify a path to a rollup config with the `--rollup.config` flag. This is used for testnets or internal deployments that are not migrated from a legacy network.
+
+Each of the above flags can also be defined via an environment variable. Run `op-node --help` to see a list of all available flags and environment variables.
+
+### Configuring Peer-to-Peer Networking
+
+Unlike the previous system, the `op-node` participates in a peer-to-peer network. This network is used to distribute blocks that have not been submitted to L1 yet. The `op-node` will automatically discover and connect to peers using a hardcoded set of bootnodes. You can also manually specify peers to connect to via the `--p2p.static` flag.
+
+For best results, run `op-node` with a static IP address that is accessible from the public Internet. For Kubernetes deployments, this can be achieved by configuring a dedicated `Ingress` with an external IP, and using the `--p2p.advertise.ip` flag to specify the IP address of the load balancer when advertising IP addresses to peers.
+
+The default port for the peer-to-peer network is `9003`. You will need to open this port on your firewall to receive unsubmitted blocks. For your node to be discoverable, this port must be accessible via both TCP and UDP protocols.
+
+## Legacy Geth
+
+If you are running a node for an upgraded network like OP Mainnet (but not OP Sepolia), you will also need to run Legacy Geth in order to serve historical execution traces. Fundamentally, Legacy Geth is our old `l2geth` binary running against a preconfigured data directory. To configure Legacy Geth, follow the instructions above for using a preconfigured data directory, then execute the following command:
+
+<Callout type="danger">
+  It is imperative that you specify the `USING_OVM=true` environment variable in the command below. Failing to specify this will cause `l2geth` to return invalid execution traces, or panic at startup.
+</Callout>
+
+```bash
+USING_OVM=true \
+  ETH1_SYNC_SERVICE_ENABLE=false \
+  RPC_API=eth,rollup,net,web3,debug \
+  RPC_ADDR=0.0.0.0 \
+  RPC_CORS_DOMAIN=* \
+  RPC_ENABLE=true \
+  RPC_PORT=8545 \
+  RPC_VHOSTS=* \
+  geth --datadir <path to data directory>
+```
+
+This command is the minimum required to run Legacy Geth and expose a functioning RPC endpoint. As before, `l2geth` takes all standard `go-ethereum` flags so you can customize the configuration as needed.
+
+As mentioned above, don't forget to specify `--rollup.historicalrpc` on `op-geth` to properly route requests for historical execution to Legacy Geth.
+
+Since Legacy Geth is read-only, it is safe to run multiple Legacy Geth nodes behind a load balancer.
+
+### Historical Execution vs. Historical Data Routing
+
+Only requests for historical execution will be routed to Legacy Geth.
+Everything else will be served by `op-geth` directly.
+The term *historical execution* refers to RPC methods that need to execute transactions prior to bedrock (not just read data from the database):
+
+*   `eth_call`
+*   `eth_estimateGas`
+*   `debug_traceBlockByNumber`
+*   `debug_traceBlockByHash`
+*   `debug_traceCall`
+*   `debug_traceTransaction`
+
+If you do not need these RPC methods for historical data, then you do not need to run Legacy Geth at all.
+
+## Next Steps
+*   See the [op-node configuration](/builders/node-operators/configuration/consensus-config) guide for additional configuration options for `op-node` and the Consensus-Layer.
+*   Similarly, visit the [op-geth configuration](/builders/node-operators/configuration/execution-config) guide for additional configuration options for `op-geth` and Execution-Layer.
+*   If you run into any problems, please reach out to our [developer support forum](https://github.com/ethereum-optimism/developers/discussions) for help.