🍫 add node on ethereums dir

This commit is contained in:
bt3gl 2022-09-29 08:53:02 +00:00
parent 48bf0f5af4
commit 54fc331be4
20 changed files with 3 additions and 7 deletions

View file

@ -8,7 +8,7 @@
* [running nodes / validators](https://github.com/bt3gl-labs/1337_mev_toolkit/blob/main/nodes/running-a-node.md)
* [running nodes / validators (advanced)](https://github.com/bt3gl-labs/1337_mev_toolkit/blob/main/nodes/running-a-node-advanced.md)
* [geth on kubernetes](geth_and_k8s)
<br>

View file

@ -0,0 +1,108 @@
# [experiment] deploy a private ethereum blockchain on kubernetes
<br>
### tl;dr
<br>
* deploy it with your own [geth](https://geth.ethereum.org/) and a customized genesis file and all.
* geth implements [Clique](https://eips.ethereum.org/EIPS/eip-225#:~:text=Clique%20is%20a%20proof%2Dof,any%20client%20with%20minimal%20effort.), a proof-of-authority mechanism to reach consensus. Moreover, a number of accounts a defined to produce new blocks in the chain.
* a custom Ethereum testnet contains three components:
* a custom [genesis file](/genesis.json
* a custom data directory, which we define with `GETH_DATADIR`
* a custom `NetworkID` , which we set `137`
<br>
---
<br>
### documentation for this work
<br>
* [genesis block explanation](https://github.com/bt3gl-labs/1337_mev_toolkit/blob/main/geth_and_k8s/genesis_block_explanation.md)
* [deployment on gcp and docker](https://github.com/bt3gl-labs/1337_mev_toolkit/blob/main/geth_and_k8s/deployment_gcp_docker.md)
* [deployment of geth in gcp + kubernetex](https://github.com/bt3gl-labs/1337_mev_toolkit/blob/main/geth_and_k8s/deployment_gcp_k8s.md)
* [running a node in ethereum with geth](https://github.com/bt3gl-labs/1337_mev_toolkit/blob/main/geth_and_k8s/running-a-node-geth.md)
* [geth creation commands](https://github.com/bt3gl-labs/1337_mev_toolkit/blob/main/geth_and_k8s/geth_creation_commands.md)
<br>
---
<br>
### nodes
<br>
* running nodes are responsible for the following tasks:
- store blockchain data,
- participates in block validation (verifying blocks and states),
- serve the network and provides data on request.
* interacting with the client:
<br>
##### getting total coinbase
```bash
web3.fromWei(eth.getBalance(eth.coinbase), "ether")
```
##### getting a balance from an account
```bash
web3.fromWei(eth.getBalance("2ee8D80de1c389f1254e94bc44D2d1Bc391eD402"), "ether")
```
##### txpool api
* access to several non-standard RPC methods to inspect the contents of the transaction pool containing all the currently pending transactions as well as the ones queued for future processing:
```bash
txpool
```
##### useful checks
```bash
admin.nodeInfo
```
##### setting this chain on metamask
* click a `custom RPC` network and add:
```bash
Network Name: "<project name>"
New RPC URL: localhost
Chain ID: 137
```
<br>
---
<br>
### references
* [geth commands](https://geth.ethereum.org/docs/interface/command-line-options)

View file

@ -0,0 +1,205 @@
# Deployment of geth in GCP + docker
In this deployment, we use **Container-Optimized OS** to permit the deployment of a Docker image to a **Google Compute Engine VM**.
This is the simplest way to run the `ethereum/client-go` image. This will install `geth`, the **Golang CLI Ethereum client** for running a full Ethereum node and interacting with Ethereum networks.
---
### Installing
Install [google cloud sdk](https://cloud.google.com/sdk/docs/quickstart) and configure it with:
```
gcloud init
gcloud auth login
gcloud config set project <project name>
```
### Create the instance
```
gcloud beta compute instances create-with-container <docker container alias> \
--boot-disk-size=500GB \
--boot-disk-type=pd-ssd \
--container-image=ethereum/client-go \
--container-arg="--rpc" \
--container-arg="--rpcaddr=0.0.0.0" \
--container-restart-policy=always \
--container-mount-host-path=\
mount-path=/root,\
host-path=/tmp/client-go,\
mode=rw \
--image-family=cos-stable \
--image-project=cos-cloud \
--zone=<which zone?> \
--project=<project name>
```
The instance should be available at the project's [GCP dashboard](https://console.cloud.google.com/compute/instances).
`Geth` needs to have a special CORS setting enabled to allow MetaMask to connect to it by default, so try starting it with this command:
```
geth --rpc --rpccorsdomain="chrome-extension://<hash>"
```
### Start `konlet-startup`
`konlet-startup` is the `systemd` service that corresponds to the docker image deployment in the **Container-Optimized OS**.
Run:
```
gcloud compute ssh <docker container alias> \
--command="sudo journalctl --unit=konlet-startup --follow" \
--project=<project name>
```
### Confirm that the container is running
```
gcloud compute ssh <docker container alias> \
--command="docker ps --format='[{{.ID}}] {{.Names}}: {{.Status}}'" \
--project=<project name>
```
### SSH and port-forwarding
```
gcloud compute ssh <docker container alias> \
--ssh-flag="-L 8545:localhost:8545" \
--project=<project name>
```
Success looks like:
```
########################[ Welcome ]########################
# You have logged in to the guest OS. #
# To access your containers use 'docker attach' command #
###########################################################
<user>@<project name> >
```
### Additional configuration for the intance
* [Alocate a static IP](https://console.cloud.google.com/networking/addresses/).
* [Prevent deletion](https://console.cloud.google.com/compute/instancesDetail/).
* Configure CORS [here]().
----
## Creating the Testnet
The command to create a testnet with `geth` is:
```
geth init --datadir data ./genesis.json
```
and then starting the node:
```
geth --datadir data --networkid NetworkID
```
`NetworkID` helps ensure the privacy of your network. You can use any number and peers joining the network must use the same number.
### Starting the genesis block
Inside the instance, create the `genesis.json` file:
```
vim ~/genesis.json
```
with
```
{
"config": {
"chainId": 137,
"homesteadBlock": 0,
"eip150Block": 0,
"eip155Block": 0,
"eip158Block": 0,
"byzantiumBlock": 0,
"constantinopleBlock": 0,
"petersburgBlock": 0
},
"difficulty": "0x400",
"gasLimit":"0x2100000",
"alloc": {
"3282791d6fd713f1e94f4bfd565eaa78b3a0599d": {
"balance": "1337000000000000000000"
}
}
}
```
Start the testnet with:
```
geth --identity <project name> --nodiscover --networkid 137 --datadir ~/data init ~/genesis.json
```
Where `geth` flags are:
* `--datadir` indicates the data for our local testnet.
* `--nodiscover`, `--maxpeers 0`, `--rpc`, `--rpcapi` make sure the network is private.
---
## Interacting with the testnet
### Connecting to a member node
From another terminal, run:
```
docker run \
--rm \
--interactive \
--net=host \
--tty \
ethereum/client-go \
attach http://localhost:8545
```
or
```
geth attach http://localhost:8545
```
Success looks like:
```
Welcome to the Geth JavaScript console!
>
```
### Monitoring
* GCP logs can be found [here](https://console.cloud.google.com/logs).
* To look at the output from the tesnet run:
```
FILTER="resource.type=\"global\" "\
"logName=\"projects/<project-name>/logs/gcplogs-docker-driver\" "
gcloud logging read "${FILTER}" \
--project=<project name> \
--format="value(jsonPayload.data)" \
--order=asc
```

View file

@ -0,0 +1,53 @@
# Deployment of geth in GCP + Kubernetes
## Deployment
```
NAMESPACE=ethereum
kubectl create namespace ${NAMESPACE}
kubectl apply --filename=deployment.yaml --namespace=${NAMESPACE}
```
## Accessing node without opening firewall
```
SERVICE=ethereum
NAMESPACE=ethereum
NODE=$(\
kubectl get nodes \
--output=jsonpath='{.items[0].metadata.name}')
PORT=$(\
kubectl get services/${SERVICE} \
--namespace=${NAMESPACE} \
--output=jsonpath='{.spec.ports[?(@.name=="default")].nodePort}')
echo ${PORT}
gcloud compute ssh <docker container alias> \
--ssh-flag="-L ${PORT}:localhost:${PORT}" \
--project=<project name>
```
## Member connecting
```
docker run \
--rm \
--interactive \
--tty \
--net=host \
ethereum/client-go attach http://localhost:${PORT}
```
### Dealing with secrets
```
kubectl create secret generic keystore \
--from-file=path/to/keystore \
--namespace=$NAMESPACE
```

View file

@ -0,0 +1,69 @@
#!/usr/bin/env bash
set -ex
GETH_DATADIR=${GETH_DATADIR:-"/data"}
geth_init() {
ls -al "$GETH_DATADIR"
if [[ -d "$GETH_DATADIR/keystore" ]]; then
echo "Chain already initialized"
return 0
fi
echo "Initializing using Genesis"
geth init --datadir "$GETH_DATADIR" /genesis.json
}
geth_custom_start() {
echo "Starting node with custom arguments $@"
exec geth $@
}
geth_normal_start() {
echo "Starting node"
local identity=${GETH_IDENTITY:-"<docker container alias>"}
local http_corsdomain=${GETH_HTTP_CORSDOMAIN:-"http://localhost:3000"}
local http_vhosts=${GETH_HTTP_VHOSTS:-"localhost"}
local networkid=${GETH_NETWORKID:-"137"}
local enable_mining=${ENABLING_MINING:-"YES"}
local mining_args=""
if [[ "$enable_mining" == "YES" ]]; then
local miner_etherbase=${GETH_MINER_ETHERBASE:-"<add etherbase>"}
local miner_threads=${GETH_MINER_THREADS:-1}
mining_args="$mining_args --mine"
mining_args="$mining_args --miner.threads $miner_threads"
mining_args="$mining_args --miner.etherbase $miner_etherbase"
fi
exec geth \
--identity "$identity" \
--nodiscover \
--http \
--http.addr 0.0.0.0 \
--http.corsdomain "$http_corsdomain" \
--http.vhosts "$http_vhosts" \
--networkid "$networkid" \
--datadir "$GETH_DATADIR" \
$mining_args \
"$@"
}
main() {
local should_initialize=${SHOULD_INITIALIZE:-"YES"}
local custom_start=${CUSTOM_START:-"NO"}
if [[ "$should_initialize" == "YES" ]]; then
geth_init
fi
if [[ "$custom_start" == "YES" ]]; then
geth_custom_start "$@"
else
geth_normal_start "$@"
fi
}
main "$@"

View file

@ -0,0 +1,8 @@
FROM ethereum/client-go:stable
RUN apk add --no-cache bash
COPY entrypoint.sh /entrypoint.sh
COPY genesis.json /genesis.json
ENTRYPOINT /entrypoint.sh

View file

@ -0,0 +1,29 @@
{
"config": {
"chainId": 137,
"homesteadBlock": 0,
"eip150Block": 0,
"eip155Block": 0,
"eip158Block": 0,
"byzantiumBlock": 0,
"constantinopleBlock": 0,
"petersburgBlock": 0
},
"difficulty": "0x400",
"gasLimit":"0x2100000",
"nonce": "0x000000000fab0042",
"alloc": {
"3282791d6fd713f1e94f4bfd565eaa78b3a0599d": {
"balance": "1337000000000000000000"
},
"64D2ea7000e831E03e6B930AC348fD90D4ACE2B8": {
"balance": "1337000000000000000000"
},
"2ee8D80de1c389f1254e94bc44D2d1Bc391eD402": {
"balance": "1337000000000000000000"
},
"Ac03BB73b6a9e108530AFf4Df5077c2B3D481e5A": {
"balance": "1337000000000000000000"
}
}
}

View file

@ -0,0 +1,30 @@
## What's the genesis block
The genesis block is a `JSON` file hardcoded into clients and Ethereum's consensus algorithm.
It's the start of the blockchain, the first block (`block 0`), and the only block that does not point to a predecessor block.
It ensures that no other node will agree unless they have the same genesis block.
---
#### Fields
* **chainId**: A unique identifier of the new private blockchain
* **mixhash**: A `256-bit` hash which proves, combined with the nonce, that a sufficient amount of computation has been carried out on this block (Proof-of-Work).
* **nonce**: A `64-bit hash`, which proves, combined with the mixhash, that a sufficient amount of computation has been carried out on this block (Proof-of-Work).
* **difficulty**: A value corresponding to the difficulty level applied during the nonce discovering of this block. The higher the difficulty, the statistically more calculations a miner must perform to discover a valid block.
* **alloc**: Allows defining a list of pre-filled wallets.
* **coinbase**: A `160-bit` address to which all rewards (in Ether) collected from the successful mining of the block have been transferred. This can be anything in the Genesis Block since the value is set by the setting of the Miner when a new Block is created.
* **timestamp**: A value equal to the reasonable output of Unix `time()` function at this block inception.
* **parentHash**: A `256-bit` hash of the entire parent block header (including its nonce and mixhash). Pointer to the parent block, thus effectively building the chain of blocks. In the case of the Genesis block, and only in this case, its 0.
* **gasLimit**: A scalar value equal to the current chain-wide limit of Gas expenditure per block. High in our case to avoid being limited by this threshold during tests.

View file

@ -0,0 +1,75 @@
# geth creation commands
### Create a miner account
This will generate a public/private key pair. By default, keys are stored inside, `<datadir>/keystore`. Everything `geth` persists gets written inside `<datadir>`.
The following command returns a private testnet address (which should be saved together with the password):
```
geth account new --datadir /path/to/data
```
### Start mining
With `NetworkID`:
```
geth --mine --rpc --networkid NetworkID --datadir /path/to/data
```
Start mining:
```
> miner.start()
```
To end mining, type:
```
> miner.stop()
```
### Pre-fund the account
Deploying contracts or making transactions requires test ether, which has no value and can be acquired for free from several "faucets".
The command `geth removedb` deletes the locally synced blockchain data of the public testnet (i.e., resyncs with the chain).
To add magic ether, remove the created blockchain database:
```
geth removedb --datadir /path/to/data
```
Modify `genesis.json` as necessary, including multiple accounts in the seed if wanted.
Finally, re-initialize the genesis block:
```
geth --identity <project name> --nodiscover --networkid 1337 --datadir /path/to/data init /path/to/genesis.json
```
### Create new accounts
```
geth --datadir . account new
```
### List accounts
```
geth --datadir . account list
```
### Connect to the testnet
```
geth attach
```
You will enter into the **Geth Javascript console**, where you can run JavaScript code.

View file

@ -0,0 +1,12 @@
---
apiVersion:
kind: Certificate
metadata:
labels:
env: staging
app:
runtime-component: web
spec:
dnsNames:
-
secretName:

View file

@ -0,0 +1,114 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
name: web
env: staging
app:
runtime-component: web
annotations:
spec:
replicas: 1
selector:
matchLabels:
name: web
template:
metadata:
labels:
name: web
env: staging
app:
annotations:
seccomp.security.alpha.kubernetes.io/pod: runtime/default
spec:
automountServiceAccountToken: true
containers:
- name: web
image:
ports:
- name: http
containerPort: 8545
envFrom:
- configMapRef:
name: application-state
env:
- name: SHOULD_INITIALIZE
value: "YES"
- name: ENABLING_MINING
value: "YES"
- name: ENV
value: staging
- name: APP_ENV
value: staging
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: NODE_IP
valueFrom:
fieldRef:
fieldPath: status.hostIP
- name: TRACE_PROXY_ADDR
value: http://$(NODE_IP):8096/
- name: STATSD_ADDR
value: 127.0.0.1:8125
- name: STATSD_IMPLEMENTATION
value: datadog
- name: IDENTITY_CLIENT_ID
valueFrom:
secretKeyRef:
name: identity-config
key: clientId
optional: true
- name: IDENTITY_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: identity-config
key: clientSecret
optional: true
resources:
limits:
cpu: 2000m
memory: 1Gi
requests:
cpu: 1500m
memory: 512Mi
volumeMounts:
- name: ejson-keys
readOnly: true
mountPath: /key
- name: data
mountPath: /data
readinessProbe:
httpGet:
port: 8545
httpHeaders:
- name: X-Forwarded-Proto
value: https
path: /
initialDelaySeconds: 20
timeoutSeconds: 3
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
- CHOWN
- DAC_OVERRIDE
- KILL
- SETGID
- SETUID
drop:
- ALL
privileged: false
volumes:
- name: ejson-keys
secret:
secretName: ejson-keys
- name: data
persistentVolumeClaim:
claimName: web
strategy:
type: Recreate
progressDeadlineSeconds: 120

View file

@ -0,0 +1,24 @@
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
labels:
name: web
env: staging
app:
runtime-component: web
annotations:
kubernetes.io/ingress.class: nginx
kubernetes.io/tls-acme: "false"
spec:
rules:
- host:
http:
paths:
- backend:
serviceName: web
servicePort: 80
tls:
- hosts:
-
secretName:

View file

@ -0,0 +1,20 @@
---
kind: RuntimeManifest
apiVersion:
metadata:
name:
runtimeInfo:
appName:
env: staging
appImage:
domains:
-
defaultBaseSource:
components:
- name: web
baseSource:
resources:
- base: deployment.yaml
- base: service.yaml
- base: certificate.yaml
- base: ingress.yaml

View file

@ -0,0 +1,18 @@
---
apiVersion: v1
kind: Service
metadata:
labels:
name: web
env: staging
app:
runtime-component: web
spec:
selector:
name: web
env: staging
app:
ports:
- name: http
port: 80
targetPort: 8545

View file

@ -0,0 +1,43 @@
## Running a node in Ethereum with `geth`
```
geth -help
```
```
COMMANDS:
account Manage accounts
attach Start an interactive JavaScript environment (connect to node)
console Start an interactive JavaScript environment
db Low level database operations
dump Dump a specific block from storage
dumpconfig Show configuration values
dumpgenesis Dumps genesis block JSON configuration to stdout
export Export blockchain into file
export-preimages Export the preimage database into an RLP stream
import Import a blockchain file
import-preimages Import the preimage database from an RLP stream
init Bootstrap and initialize a new genesis block
js Execute the specified JavaScript files
license Display license information
makecache Generate ethash verification cache (for testing)
makedag Generate ethash mining DAG (for testing)
removedb Remove blockchain and state databases
show-deprecated-flags Show flags that have been deprecated
snapshot A set of commands based on the snapshot
version Print version numbers
version-check Checks (online) whether the current version suffers from any known security vulnerabilities
wallet Manage Ethereum presale wallets
```
Advanced options are:
```
ETHEREUM OPTIONS
LIGHT CLIENT OPTIONS
DEVELOPER CHAIN OPTIONS
ETHASH OPTIONS
```

View file

@ -0,0 +1,17 @@
#!/usr/bin/env bash
set -ex
PORT=${PORT:-18545}
echo "Building local blockchain container"
docker build -t <docker container alias> .
echo "Starting blockchain network on port $PORT (use rpc URL http://localhost:$PORT)"
docker run -it --rm \
-v $(pwd)/genesis.json:/genesis.json \
-v $(pwd)/data:/data \
-p $PORT:8545 \
<docker container alias>

View file

@ -0,0 +1,69 @@
#!/usr/bin/env bash
set -ex
GETH_DATADIR=${GETH_DATADIR:-"/data"}
geth_init() {
ls -al "$GETH_DATADIR"
if [[ -d "$GETH_DATADIR/keystore" ]]; then
echo "Chain already initialized"
return 0
fi
echo "Initializing using Genesis"
geth init --datadir "$GETH_DATADIR" /genesis.json
}
geth_custom_start() {
echo "Starting node with custom arguments $@"
exec geth $@
}
geth_normal_start() {
echo "Starting node"
local identity=${GETH_IDENTITY:-"<docker container alias>"}
local http_corsdomain=${GETH_HTTP_CORSDOMAIN:-"http://localhost:3000"}
local http_vhosts=${GETH_HTTP_VHOSTS:-"localhost"}
local networkid=${GETH_NETWORKID:-"137"}
local enable_mining=${ENABLING_MINING:-"YES"}
local mining_args=""
if [[ "$enable_mining" == "YES" ]]; then
local miner_etherbase=${GETH_MINER_ETHERBASE:-"<add etherbase>"}
local miner_threads=${GETH_MINER_THREADS:-1}
mining_args="$mining_args --mine"
mining_args="$mining_args --miner.threads $miner_threads"
mining_args="$mining_args --miner.etherbase $miner_etherbase"
fi
exec geth \
--identity "$identity" \
--nodiscover \
--http \
--http.addr 0.0.0.0 \
--http.corsdomain "$http_corsdomain" \
--http.vhosts "$http_vhosts" \
--networkid "$networkid" \
--datadir "$GETH_DATADIR" \
$mining_args \
"$@"
}
main() {
local should_initialize=${SHOULD_INITIALIZE:-"YES"}
local custom_start=${CUSTOM_START:-"NO"}
if [[ "$should_initialize" == "YES" ]]; then
geth_init
fi
if [[ "$custom_start" == "YES" ]]; then
geth_custom_start "$@"
else
geth_normal_start "$@"
fi
}
main "$@"

View file

@ -0,0 +1,8 @@
FROM ethereum/client-go:stable
RUN apk add --no-cache bash
COPY entrypoint.sh /entrypoint.sh
COPY genesis.json /genesis.json
ENTRYPOINT /entrypoint.sh

View file

@ -0,0 +1,29 @@
{
"config": {
"chainId": 137,
"homesteadBlock": 0,
"eip150Block": 0,
"eip155Block": 0,
"eip158Block": 0,
"byzantiumBlock": 0,
"constantinopleBlock": 0,
"petersburgBlock": 0
},
"difficulty": "0x400",
"gasLimit":"0x2100000",
"nonce": "0x000000000fab0042",
"alloc": {
"3282791d6fd713f1e94f4bfd565eaa78b3a0599d": {
"balance": "1337000000000000000000"
},
"64D2ea7000e831E03e6B930AC348fD90D4ACE2B8": {
"balance": "1337000000000000000000"
},
"2ee8D80de1c389f1254e94bc44D2d1Bc391eD402": {
"balance": "1337000000000000000000"
},
"Ac03BB73b6a9e108530AFf4Df5077c2B3D481e5A": {
"balance": "1337000000000000000000"
}
}
}