Skip to content
This repository was archived by the owner on Feb 16, 2020. It is now read-only.

Commit ec790cf

Browse files
tlextlex
committed
Merge branch 'patch-1' into 'master'
Overall improvements * Closes #3 Closes #6: Restructure. Adds deprecation for `source_currency` * Closes #7: Formats the variables in README.md * Closes #1: Updates the link to the image to point to docker hub * Closes #4: Switches to f-Strings instead of `.format()` * Closes #5: Adds VERSION and BUILD instead of just VERSION made of both See merge request https://gitlab.com/ix.ai/etherscan-exporter/-/merge_requests/11
2 parents bb7bd35 + f8fe5d2 commit ec790cf

File tree

15 files changed

+256
-191
lines changed

15 files changed

+256
-191
lines changed

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
__pycache__
2+
.local

‎Dockerfile‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,15 @@ FROM alpine:latest
22
LABEL maintainer="[email protected]" \
33
ai.ix.repository="ix.ai/etherscan-exporter"
44

5-
WORKDIR /app
6-
7-
COPY src/ /app
5+
COPY etherscan-exporter/requirements.txt /etherscan-exporter/requirements.txt
86

97
RUN apk --no-cache upgrade && \
108
apk --no-cache add python3 py3-pip py3-requests py3-prometheus-client && \
11-
pip3 install --no-cache-dir -r requirements.txt
9+
pip3 install --no-cache-dir -r /etherscan-exporter/requirements.txt
10+
11+
COPY etherscan-exporter/ /etherscan-exporter/
12+
COPY etherscan-exporter.sh /usr/local/bin/etherscan-exporter.sh
1213

1314
EXPOSE 9308
1415

15-
ENTRYPOINT ["python3", "/app/etherscan-exporter.py"]
16+
ENTRYPOINT ["/usr/local/bin/etherscan-exporter.sh"]

‎README.md‎

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -15,18 +15,21 @@ docker run --rm -it -p 9308:9308 \
1515
-e ADDRESSES="0x90833394dB1b53f08B9D97dab8BEFf69FCf3bA49" \
1616
-e TOKENS='[{"contract":"0x9b70740e708a083c6ff38df52297020f5dfaa5ee","name":"Daneel","short":"DAN","decimals": 10},{"contract":"0xd26114cd6EE289AccF82350c8d8487fedB8A0C07","name":"OmiseGO","short":"OMG","decimals":18}]'
1717
--name etherscan-exporter \
18-
registry.gitlab.com/ix.ai/etherscan-exporter:latest
18+
ixdotai/etherscan-exporter:latest
1919
```
2020

2121
## Supported variables:
22-
* `API_KEY` (no default - **mandatory**) - set this to your Etherscan API key
23-
* `URL` (default: https://api.etherscan.io/api) - set this to your Etherscan API secret
24-
* `ADDRESSES` (no default) - a comma separated list of the ETH addresses to export
25-
* `TOKENS` (no default) - a JSON object with the list of tokens to export (see below)
26-
* `GELF_HOST` (no default) - if set, the exporter will also log to this [GELF](https://docs.graylog.org/en/3.0/pages/gelf.html) capable host on UDP
27-
* `GELF_PORT` (defaults to `12201`) - the port to use for GELF logging
28-
* `PORT` (defaults to `9308`) - the listen port for the exporter
29-
* `LOGLEVEL` (defaults to `INFO`)
22+
23+
| **Variable** | **Default** | **Mandatory** | **Description** |
24+
|:-------------|:------------------------------:|:-------------:|:-----------------|
25+
| `API_KEY` | - | **YES** | set this to your Etherscan API key |
26+
| `URL` | `https://api.etherscan.io/api` | NO | The URL of the Etherscan API |
27+
| `ADDRESSES` | - | NO | A comma separated list of the ETH addresses to export |
28+
| `TOKENS` | - | NO | A JSON object with the list of tokens to export (see [below](#tokens-variable)) |
29+
| `LOGLEVEL` | `INFO` | NO | [Logging Level](https://docs.python.org/3/library/logging.html#levels) |
30+
| `GELF_HOST` | - | NO | If set, the exporter will also log to this [GELF](https://docs.graylog.org/en/3.0/pages/gelf.html) capable host on UDP |
31+
| `GELF_PORT` | `12201` | NO | Ignored, if `GELF_HOST` is unset. The UDP port for GELF logging |
32+
| `PORT` | `9308` | NO | The port for prometheus metrics |
3033

3134
## Tokens variable:
3235
Example:
@@ -41,7 +44,6 @@ The technical information can be found on [etherscan.io](https://etherscan.io/to
4144
Starting with version v0.3.1, the images are multi-arch, with builds for amd64, arm64, armv7 and armv6.
4245
* `vN.N.N` - for example v0.3.0
4346
* `latest` - always pointing to the latest version
44-
* `dev-branch` - the last build on a feature/development branch
4547
* `dev-master` - the last build on the master branch
4648

4749
## Resources:

‎build.sh‎

100644100755
Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
#!/usr/bin/env sh
22

3-
echo "Setting VERSION='${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHORT_SHA}' in src/constants.py"
4-
echo "VERSION = '${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHORT_SHA}'" >> src/constants.py
3+
echo "Setting VERSION"
4+
find . -name .git -type d -prune -o -type f -name constants.py -exec sed -i s/^VERSION.*/VERSION\ =\ \'${CI_COMMIT_REF_NAME}\'/g {} + -exec grep VERSION {} +
5+
echo "Setting BUILD"
6+
find . -name .git -type d -prune -o -type f -name constants.py -exec sed -i s/^BUILD.*/BUILD\ =\ \'${CI_COMMIT_SHORT_SHA}\'/g {} + -exec grep BUILD {} +

‎etherscan-exporter.sh‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
#!/usr/bin/env sh
2+
3+
exec python3 -m "etherscan-exporter.etherscan-exporter" "$@"

‎etherscan-exporter/__init__.py‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
#!/usr/bin/env python3
2+
# -*- coding: utf-8 -*-
3+
""" initializes etherscan-exporter """
4+
5+
import os
6+
from .lib import log as logging
7+
8+
log = logging.setup_logger(
9+
name=__package__,
10+
level=os.environ.get('LOGLEVEL', 'INFO'),
11+
gelf_host=os.environ.get('GELF_HOST'),
12+
gelf_port=int(os.environ.get('GELF_PORT', 12201)),
13+
_ix_id=__package__,
14+
)

‎etherscan-exporter/etherscan-exporter.py‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
#!/usr/bin/env python3
2+
"""Prometheus Exporter for Etherscan"""
3+
4+
import logging
5+
import time
6+
import os
7+
from prometheus_client import start_http_server
8+
from prometheus_client.core import REGISTRY
9+
from .lib import constants
10+
from .etherscan_collector import EtherscanCollector
11+
from .etherscan import Etherscan
12+
13+
14+
log = logging.getLogger(__package__)
15+
version = f'{constants.VERSION}-{constants.BUILD}'
16+
17+
18+
if __name__ == '__main__':
19+
port = int(os.environ.get('PORT', 9308))
20+
log.warning(f'Starting {__package__} {version} on port {port}')
21+
log.warning("The label 'source_currency' is deprecated and will likely be removed soon")
22+
23+
etherscan = Etherscan()
24+
REGISTRY.register(EtherscanCollector(etherscan))
25+
start_http_server(port)
26+
while True:
27+
time.sleep(1)

‎etherscan-exporter/etherscan.py‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
#!/usr/bin/env python3
2+
"""Prometheus Exporter for Etherscan"""
3+
4+
import logging
5+
import time
6+
import os
7+
import json
8+
import requests
9+
10+
log = logging.getLogger(__package__)
11+
12+
13+
class Etherscan:
14+
""" The EtherscanCollector class """
15+
16+
accounts = {}
17+
tokens = {}
18+
19+
def __init__(self):
20+
self.settings = {
21+
'api_key': os.environ.get("API_KEY"),
22+
'url': os.environ.get("URL", 'https://api.etherscan.io/api'),
23+
'addresses': os.environ.get("ADDRESSES"),
24+
'tokens': [],
25+
}
26+
if not self.settings.get('api_key'):
27+
raise ValueError("Missing API_KEY environment variable.")
28+
29+
if os.environ.get('TOKENS'):
30+
self.settings['tokens'] = (json.loads(os.environ.get("TOKENS")))
31+
32+
def get_tokens(self):
33+
""" Gets the tokens from an account """
34+
log.info('Retrieving the tokens')
35+
for account in self.accounts:
36+
for token in self.settings['tokens']:
37+
log.debug(f"Retrieving the balance for {token['short']} on the account {account}")
38+
request_data = {
39+
'module': 'account',
40+
'action': 'tokenbalance',
41+
'contractaddress': token['contract'],
42+
'address': account,
43+
'tag': 'latest',
44+
'apikey': self.settings['api_key'],
45+
}
46+
decimals = 18
47+
if token.get('decimals', -1) >= 0:
48+
decimals = int(token['decimals'])
49+
try:
50+
req = requests.get(self.settings['url'], params=request_data).json()
51+
except (
52+
requests.exceptions.ConnectionError,
53+
requests.exceptions.ReadTimeout,
54+
) as error:
55+
log.exception(f'Exception caught: {error}')
56+
req = {}
57+
if req.get('result') and int(req['result']) > 0:
58+
self.tokens.update({
59+
f"{account}-{token['short']}": {
60+
'account': account,
61+
'name': token['name'],
62+
'name_short': token['short'],
63+
'contract_address': token['contract'],
64+
'value': int(req['result']) / (10**decimals) if decimals > 0 else int(req['result'])
65+
}
66+
})
67+
time.sleep(1) # Ensure that we don't get rate limited
68+
log.debug(f'Tokens: {self.tokens}')
69+
return self.tokens
70+
71+
def get_balances(self):
72+
""" Gets the current balance for an account """
73+
log.info('Retrieving the account balances')
74+
request_data = {
75+
'module': 'account',
76+
'action': 'balancemulti',
77+
'address': self.settings['addresses'],
78+
'tag': 'latest',
79+
'apikey': self.settings['api_key'],
80+
}
81+
try:
82+
req = requests.get(self.settings['url'], params=request_data).json()
83+
except (
84+
requests.exceptions.ConnectionError,
85+
requests.exceptions.ReadTimeout,
86+
) as error:
87+
log.exception(f'Exception caught: {error}')
88+
req = {}
89+
if req.get('message') == 'OK' and req.get('result'):
90+
for result in req.get('result'):
91+
self.accounts.update({
92+
result['account']: float(result['balance'])/(1000000000000000000)
93+
})
94+
log.debug(f'Accounts: {self.accounts}')
95+
return self.accounts

‎etherscan-exporter/etherscan_collector.py‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
#!/usr/bin/env python3
2+
"""Prometheus Exporter for Etherscan"""
3+
4+
import logging
5+
from prometheus_client.core import GaugeMetricFamily
6+
7+
log = logging.getLogger(__package__)
8+
9+
10+
class EtherscanCollector:
11+
""" The EtherscanCollector class """
12+
13+
accounts = {}
14+
tokens = {}
15+
16+
def __init__(self, etherscan):
17+
self.etherscan = etherscan
18+
19+
def describe(self):
20+
""" Just a needed method, so that collect() isn't called at startup """
21+
return []
22+
23+
def collect(self):
24+
"""The method that actually does the collecting"""
25+
metrics = {
26+
'account_balance': GaugeMetricFamily(
27+
'account_balance',
28+
'Account Balance',
29+
labels=['source_currency', 'currency', 'account', 'type']
30+
),
31+
}
32+
accounts = self.etherscan.get_balances()
33+
for account in accounts:
34+
metrics['account_balance'].add_metric(
35+
value=(accounts[account]),
36+
labels=[
37+
'ETH',
38+
'ETH',
39+
account,
40+
'etherscan'
41+
]
42+
)
43+
44+
tokens = self.etherscan.get_tokens()
45+
for token in tokens:
46+
metrics['account_balance'].add_metric(
47+
value=(tokens[token]['value']),
48+
labels=[
49+
tokens[token]['name_short'],
50+
tokens[token]['name_short'],
51+
tokens[token]['account'],
52+
'etherscan'
53+
]
54+
)
55+
for metric in metrics.values():
56+
yield metric

‎etherscan-exporter/lib/__init__.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
#!/usr/bin/env python3
2+
# -*- coding: utf-8 -*-

0 commit comments

Comments
 (0)