Skip to content

Commit e1704ef

Browse files
1yamhohPsycojokerolethanhnesitor
authored
Feature: Allow User to control their VM (#124)
* Feature: VmClient * Fix: Protocol (http/https) should not be hardcoded. One place hardcoded `http://`, the other one `https://`. * Fix: There was no test for `notify_allocation()`. * WIP: Copy authentication functions from aleph-vm * Fix: vm client sessions wasn't close + authentifications for test will use localhost as domain * Add: Unit test for {perform_operation, stop, reboot, erase, expire} * Refactor: logs didn't need to generate full header Fix: extracts domain from node url instead of sending url Fix: using vmclient sessions in get_logs instead of creating new one * Add: get_logs test * Fix: black in aleph_vm_authentification.py fix: isort issue Fix: mypy issue Fix: black Fix: isort * Fix: fully remove _generate_header call in get_logs Fix Fix: using real path server instead of fake server for test Fix: create playload * Fix: black issue * Fix: test fix workflow * feat(vm_client): add missing types annotations * refactor(vm_client): remove duplicated types annotations * refactor(vm_client): avoid using single letter variable names * feat(vm_client): increase test_notify_allocation precision * refactor(vm_client): add empty lines for code readability * style: run linting:fmt * Fix: Required an old version of `aleph-message` * Fix: Newer aleph-message requires InstanceEnvironment Else tests were breaking. * Fix: Qemu was not the default hypervisor for instances. * Fix: Pythom 3.12 fails setup libsecp256k1 When "using bundled libsecp256k1", the setup using `/tmp/venv/bin/hatch run testing:test` fails to proceed on Python 3.12. That library `secp256k1` has been unmaintained for more than 2 years now (0.14.0, Nov 6, 2021), and seems to not support Python 3.12. The error in the logs: ``` File "/tmp/pip-build-env-ye8d6ort/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 862, in get_command_obj cmd_obj = self.command_obj[command] = klass(self) ^^^^^^^^^^^ TypeError: 'NoneType' object is not callable [end of output] ``` See failing CI run: https://github.com/aleph-im/aleph-sdk-python/actions/runs/9613634583/job/26516767722 * doc(README): command to launch tests was incorrect * Refactor: create and sign playload goes to utils and some fix * Fix: linting issue * Fix: mypy issue * fix: black * feat: use bytes_from_hex where it makes sens * chore: use ruff new CLI api * feat: add unit tests for authentication mechanisms of VmClient * fix: debug code remove * Update vmclient.py Co-authored-by: Olivier Le Thanh Duong <[email protected]> * Fix: update unit test to use stream_logs endpoint instead of logs * Implement `VmConfidentialClient` class (#138) * Problem: A user cannot initialize an already created confidential VM. Solution: Implement `VmConfidentialClient` class to be able to initialize and interact with confidential VMs. * Problem: Auth was not working Corrections: * Measurement type returned was missing field needed for validation of measurements * Port number was not handled correctly in authentifaction * Adapt to new auth protocol where domain is moved to the operation field (While keeping compat with the old format) * Get measurement was not working since signed with the wrong method * inject_secret was not sending a json * Websocked auth was sending a twice serialized json * update 'vendorized' aleph-vm auth file from source Co-authored-by: Hugo Herter <[email protected]> Co-authored-by: Laurent Peuch <[email protected]> Co-authored-by: Olivier Le Thanh Duong <[email protected]> Co-authored-by: nesitor <[email protected]>
1 parent 8981935 commit e1704ef

File tree

12 files changed

+1447
-19
lines changed

12 files changed

+1447
-19
lines changed

pyproject.toml

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,10 @@ dependencies = [
2828
"coincurve>=19.0.0; python_version>=\"3.11\"",
2929
"eth_abi>=4.0.0; python_version>=\"3.11\"",
3030
"eth_account>=0.4.0,<0.11.0",
31+
"jwcrypto==1.5.6",
3132
"python-magic",
3233
"typing_extensions",
34+
"aioresponses>=0.7.6"
3335
]
3436

3537
[project.optional-dependencies]
@@ -121,6 +123,8 @@ dependencies = [
121123
"pytest-cov==4.1.0",
122124
"pytest-mock==3.12.0",
123125
"pytest-asyncio==0.23.5",
126+
"pytest-aiohttp==1.0.5",
127+
"aioresponses==0.7.6",
124128
"fastapi",
125129
"httpx",
126130
"secp256k1",
@@ -149,13 +153,13 @@ dependencies = [
149153
[tool.hatch.envs.linting.scripts]
150154
typing = "mypy --config-file=pyproject.toml {args:} ./src/ ./tests/ ./examples/"
151155
style = [
152-
"ruff {args:.} ./src/ ./tests/ ./examples/",
156+
"ruff check {args:.} ./src/ ./tests/ ./examples/",
153157
"black --check --diff {args:} ./src/ ./tests/ ./examples/",
154158
"isort --check-only --profile black {args:} ./src/ ./tests/ ./examples/",
155159
]
156160
fmt = [
157161
"black {args:} ./src/ ./tests/ ./examples/",
158-
"ruff --fix {args:.} ./src/ ./tests/ ./examples/",
162+
"ruff check --fix {args:.} ./src/ ./tests/ ./examples/",
159163
"isort --profile black {args:} ./src/ ./tests/ ./examples/",
160164
"style",
161165
]

src/aleph/sdk/chains/common.py

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -170,10 +170,3 @@ def get_fallback_private_key(path: Optional[Path] = None) -> bytes:
170170
if not default_key_path.exists():
171171
default_key_path.symlink_to(path)
172172
return private_key
173-
174-
175-
def bytes_from_hex(hex_string: str) -> bytes:
176-
if hex_string.startswith("0x"):
177-
hex_string = hex_string[2:]
178-
hex_string = bytes.fromhex(hex_string)
179-
return hex_string

src/aleph/sdk/chains/ethereum.py

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -7,12 +7,8 @@
77
from eth_keys.exceptions import BadSignature as EthBadSignatureError
88

99
from ..exceptions import BadSignatureError
10-
from .common import (
11-
BaseAccount,
12-
bytes_from_hex,
13-
get_fallback_private_key,
14-
get_public_key,
15-
)
10+
from ..utils import bytes_from_hex
11+
from .common import BaseAccount, get_fallback_private_key, get_public_key
1612

1713

1814
class ETHAccount(BaseAccount):

src/aleph/sdk/chains/substrate.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,8 @@
99

1010
from ..conf import settings
1111
from ..exceptions import BadSignatureError
12-
from .common import BaseAccount, bytes_from_hex, get_verification_buffer
12+
from ..utils import bytes_from_hex
13+
from .common import BaseAccount, get_verification_buffer
1314

1415
logger = logging.getLogger(__name__)
1516

src/aleph/sdk/client/vm_client.py

Lines changed: 192 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
import datetime
2+
import json
3+
import logging
4+
from typing import Any, AsyncGenerator, Dict, List, Optional, Tuple
5+
from urllib.parse import urlparse
6+
7+
import aiohttp
8+
from aleph_message.models import ItemHash
9+
from eth_account.messages import encode_defunct
10+
from jwcrypto import jwk
11+
12+
from aleph.sdk.types import Account
13+
from aleph.sdk.utils import (
14+
create_vm_control_payload,
15+
sign_vm_control_payload,
16+
to_0x_hex,
17+
)
18+
19+
logger = logging.getLogger(__name__)
20+
21+
22+
class VmClient:
23+
account: Account
24+
ephemeral_key: jwk.JWK
25+
node_url: str
26+
pubkey_payload: Dict[str, Any]
27+
pubkey_signature_header: str
28+
session: aiohttp.ClientSession
29+
30+
def __init__(
31+
self,
32+
account: Account,
33+
node_url: str = "",
34+
session: Optional[aiohttp.ClientSession] = None,
35+
):
36+
self.account = account
37+
self.ephemeral_key = jwk.JWK.generate(kty="EC", crv="P-256")
38+
self.node_url = node_url
39+
self.pubkey_payload = self._generate_pubkey_payload()
40+
self.pubkey_signature_header = ""
41+
self.session = session or aiohttp.ClientSession()
42+
43+
def _generate_pubkey_payload(self) -> Dict[str, Any]:
44+
return {
45+
"pubkey": json.loads(self.ephemeral_key.export_public()),
46+
"alg": "ECDSA",
47+
"domain": self.node_domain,
48+
"address": self.account.get_address(),
49+
"expires": (
50+
datetime.datetime.utcnow() + datetime.timedelta(days=1)
51+
).isoformat()
52+
+ "Z",
53+
}
54+
55+
async def _generate_pubkey_signature_header(self) -> str:
56+
pubkey_payload = json.dumps(self.pubkey_payload).encode("utf-8").hex()
57+
signable_message = encode_defunct(hexstr=pubkey_payload)
58+
buffer_to_sign = signable_message.body
59+
60+
signed_message = await self.account.sign_raw(buffer_to_sign)
61+
pubkey_signature = to_0x_hex(signed_message)
62+
63+
return json.dumps(
64+
{
65+
"sender": self.account.get_address(),
66+
"payload": pubkey_payload,
67+
"signature": pubkey_signature,
68+
"content": {"domain": self.node_domain},
69+
}
70+
)
71+
72+
async def _generate_header(
73+
self, vm_id: ItemHash, operation: str, method: str
74+
) -> Tuple[str, Dict[str, str]]:
75+
payload = create_vm_control_payload(
76+
vm_id, operation, domain=self.node_domain, method=method
77+
)
78+
signed_operation = sign_vm_control_payload(payload, self.ephemeral_key)
79+
80+
if not self.pubkey_signature_header:
81+
self.pubkey_signature_header = (
82+
await self._generate_pubkey_signature_header()
83+
)
84+
85+
headers = {
86+
"X-SignedPubKey": self.pubkey_signature_header,
87+
"X-SignedOperation": signed_operation,
88+
}
89+
90+
path = payload["path"]
91+
return f"{self.node_url}{path}", headers
92+
93+
@property
94+
def node_domain(self) -> str:
95+
domain = urlparse(self.node_url).hostname
96+
if not domain:
97+
raise Exception("Could not parse node domain")
98+
return domain
99+
100+
async def perform_operation(
101+
self, vm_id: ItemHash, operation: str, method: str = "POST"
102+
) -> Tuple[Optional[int], str]:
103+
if not self.pubkey_signature_header:
104+
self.pubkey_signature_header = (
105+
await self._generate_pubkey_signature_header()
106+
)
107+
108+
url, header = await self._generate_header(
109+
vm_id=vm_id, operation=operation, method=method
110+
)
111+
112+
try:
113+
async with self.session.request(
114+
method=method, url=url, headers=header
115+
) as response:
116+
response_text = await response.text()
117+
return response.status, response_text
118+
119+
except aiohttp.ClientError as e:
120+
logger.error(f"HTTP error during operation {operation}: {str(e)}")
121+
return None, str(e)
122+
123+
async def get_logs(self, vm_id: ItemHash) -> AsyncGenerator[str, None]:
124+
if not self.pubkey_signature_header:
125+
self.pubkey_signature_header = (
126+
await self._generate_pubkey_signature_header()
127+
)
128+
129+
payload = create_vm_control_payload(
130+
vm_id, "stream_logs", method="get", domain=self.node_domain
131+
)
132+
signed_operation = sign_vm_control_payload(payload, self.ephemeral_key)
133+
path = payload["path"]
134+
ws_url = f"{self.node_url}{path}"
135+
136+
async with self.session.ws_connect(ws_url) as ws:
137+
auth_message = {
138+
"auth": {
139+
"X-SignedPubKey": json.loads(self.pubkey_signature_header),
140+
"X-SignedOperation": json.loads(signed_operation),
141+
}
142+
}
143+
await ws.send_json(auth_message)
144+
145+
async for msg in ws: # msg is of type aiohttp.WSMessage
146+
if msg.type == aiohttp.WSMsgType.TEXT:
147+
yield msg.data
148+
elif msg.type == aiohttp.WSMsgType.ERROR:
149+
break
150+
151+
async def start_instance(self, vm_id: ItemHash) -> Tuple[int, str]:
152+
return await self.notify_allocation(vm_id)
153+
154+
async def stop_instance(self, vm_id: ItemHash) -> Tuple[Optional[int], str]:
155+
return await self.perform_operation(vm_id, "stop")
156+
157+
async def reboot_instance(self, vm_id: ItemHash) -> Tuple[Optional[int], str]:
158+
return await self.perform_operation(vm_id, "reboot")
159+
160+
async def erase_instance(self, vm_id: ItemHash) -> Tuple[Optional[int], str]:
161+
return await self.perform_operation(vm_id, "erase")
162+
163+
async def expire_instance(self, vm_id: ItemHash) -> Tuple[Optional[int], str]:
164+
return await self.perform_operation(vm_id, "expire")
165+
166+
async def notify_allocation(self, vm_id: ItemHash) -> Tuple[int, str]:
167+
json_data = {"instance": vm_id}
168+
169+
async with self.session.post(
170+
f"{self.node_url}/control/allocation/notify", json=json_data
171+
) as session:
172+
form_response_text = await session.text()
173+
174+
return session.status, form_response_text
175+
176+
async def manage_instance(
177+
self, vm_id: ItemHash, operations: List[str]
178+
) -> Tuple[int, str]:
179+
for operation in operations:
180+
status, response = await self.perform_operation(vm_id, operation)
181+
if status != 200 and status:
182+
return status, response
183+
return 200, "All operations completed successfully"
184+
185+
async def close(self):
186+
await self.session.close()
187+
188+
async def __aenter__(self):
189+
return self
190+
191+
async def __aexit__(self, exc_type, exc_value, traceback):
192+
await self.close()

0 commit comments

Comments
 (0)