Skip to content

GA-163 | test_multigraph & test_multidigraph #42

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 81 commits into from
Aug 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
b315606
GA-163 | initial commit
aMahanna Aug 16, 2024
aa99026
Merge branch 'main' into GA-163
aMahanna Aug 16, 2024
26e1a85
unlock adbnx
aMahanna Aug 16, 2024
d6da2a3
fix: `incoming_graph_data`
aMahanna Aug 16, 2024
6561949
fix: incoming_graph_data
aMahanna Aug 17, 2024
469de71
fix: off-by-one IDs
aMahanna Aug 17, 2024
b742027
checkpoint
aMahanna Aug 17, 2024
8b47e4d
checkpoint: `BaseGraphTester` is passing
aMahanna Aug 17, 2024
0483486
checkpoint: BaseGraphAttrTester
aMahanna Aug 18, 2024
1ed111e
cleanup: `aql_fetch_data`, `aql_fetch_data_edge`
aMahanna Aug 18, 2024
f5963a6
use pytest skip for failing tests
aMahanna Aug 18, 2024
eb6717e
checkpoint: optimize `__iter__`
aMahanna Aug 18, 2024
04dc9c1
checkpoint: run `test_graph`
aMahanna Aug 19, 2024
2199ae3
add comment
aMahanna Aug 19, 2024
173f0a7
Merge branch 'main' into GA-163
aMahanna Aug 19, 2024
bc64fe9
checkpoint
aMahanna Aug 19, 2024
0df6c2b
attempt: slleep
aMahanna Aug 19, 2024
aa4b336
fix: lint
aMahanna Aug 19, 2024
5aa3eb2
cleanup: getitem
aMahanna Aug 20, 2024
b03f4cf
cleanup: copy
aMahanna Aug 20, 2024
27adfa3
attempt: shorten sleep
aMahanna Aug 20, 2024
c34898a
Merge branch 'main' into GA-163
aMahanna Aug 20, 2024
0d18563
fix: `__set_adj_elements`
aMahanna Aug 20, 2024
b0434a9
fix: mypy
aMahanna Aug 20, 2024
3f07ae1
attempt: decrease sleep
aMahanna Aug 20, 2024
8b87046
Merge branch 'main' into GA-163
aMahanna Aug 20, 2024
0c48931
GA-163 | `test_digraph`
aMahanna Aug 21, 2024
5907d8e
checkpoint
aMahanna Aug 21, 2024
de17587
fix: set `self.Graph`
aMahanna Aug 21, 2024
3144429
add type ignore
aMahanna Aug 21, 2024
28dd130
Merge branch 'main' into GA-163
aMahanna Aug 21, 2024
443d436
Merge branch 'main' into GA-163
aMahanna Aug 21, 2024
dcb94ff
fix: graph name
aMahanna Aug 21, 2024
0fc2568
Merge branch 'GA-163' into GA-163-digraph
aMahanna Aug 21, 2024
5264178
fix: graph name
aMahanna Aug 21, 2024
b805419
adjust assertions to exclude _rev, set `use_experimental_views`
aMahanna Aug 21, 2024
c75b758
Revert "adjust assertions to exclude _rev, set `use_experimental_views`"
aMahanna Aug 21, 2024
e7339de
fix: `_rev`, `use_experimental_views`
aMahanna Aug 21, 2024
f79fabb
Merge branch 'GA-163' into GA-163-digraph
aMahanna Aug 21, 2024
4c961d0
set `use_experimental_views`
aMahanna Aug 21, 2024
058d687
fix: lint
aMahanna Aug 21, 2024
6e5b504
new: `nbunch_iter` override
aMahanna Aug 21, 2024
dd17bb5
Merge branch 'GA-163' into GA-163-digraph
aMahanna Aug 22, 2024
e64781e
set experimental views to false
aMahanna Aug 22, 2024
f19839e
Merge branch 'GA-163' into GA-163-digraph
aMahanna Aug 22, 2024
6bf6a0a
set experimental views to false
aMahanna Aug 22, 2024
27dd792
cleanup
aMahanna Aug 22, 2024
dad8365
GA-163 | `test_multigraph` checkpoint
aMahanna Aug 22, 2024
1e5ebc4
fix lint
aMahanna Aug 22, 2024
69489a7
Merge branch 'main' into GA-163-digraph
aMahanna Aug 23, 2024
a0bc891
Merge branch 'GA-163-digraph' into GA-163-multigraph
aMahanna Aug 23, 2024
b3bb830
fix: `function.py`
aMahanna Aug 23, 2024
4dca157
Merge branch 'GA-163-digraph' into GA-163-multigraph
aMahanna Aug 23, 2024
767301f
cleanup: `graph`, `digraph`
aMahanna Aug 23, 2024
9d30054
fix: `test_data_input`
aMahanna Aug 23, 2024
25b6418
attempt: wait for CircleCI
aMahanna Aug 23, 2024
6cdf2b8
fix: nx graph
aMahanna Aug 23, 2024
3b3b09c
remove sleep
aMahanna Aug 23, 2024
b23d8cd
Merge branch 'GA-163-digraph' into GA-163-multigraph
aMahanna Aug 23, 2024
df237dd
new: `override` suffix
aMahanna Aug 23, 2024
4bb97fe
Merge branch 'GA-163-digraph' into GA-163-multigraph
aMahanna Aug 23, 2024
68963b0
add override
aMahanna Aug 23, 2024
6789e6c
enable more tests
aMahanna Aug 26, 2024
34af3fc
fix: lint
aMahanna Aug 26, 2024
c5010d8
Merge branch 'GA-163-digraph' into GA-163-multigraph
aMahanna Aug 26, 2024
a87db36
checkpoint
aMahanna Aug 27, 2024
ab7223b
checkpoint: 2 remaining test failures
aMahanna Aug 27, 2024
dc354e3
Merge branch 'main' into GA-163-multigraph
aMahanna Aug 28, 2024
2626c61
fix: lint
aMahanna Aug 28, 2024
90dbdee
checkpoint: one last failing test
aMahanna Aug 28, 2024
768880d
remove: `logger_debug`, fix lint
aMahanna Aug 28, 2024
fce0c30
lint
hkernbach Aug 29, 2024
3ba811f
fix: `test_multigraph`
aMahanna Aug 29, 2024
079e62b
cleanup, add missing test
aMahanna Aug 29, 2024
43a0dd5
new: `test_non_multigraph_input_a`
aMahanna Aug 29, 2024
ea395a7
add comments
aMahanna Aug 29, 2024
ae63c72
GA-163 | `test_multidigraph` (#45)
aMahanna Aug 30, 2024
93723bb
remove unused block
aMahanna Aug 30, 2024
b40328e
Merge branch 'main' into GA-163-multigraph
aMahanna Aug 30, 2024
a4346a1
fix: `write_async` False
aMahanna Aug 30, 2024
6999620
Merge branch 'main' into GA-163-multigraph
aMahanna Aug 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
160 changes: 69 additions & 91 deletions nx_arangodb/classes/dict/adj.py

Large diffs are not rendered by default.

18 changes: 2 additions & 16 deletions nx_arangodb/classes/dict/graph.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@
json_serializable,
key_is_not_reserved,
key_is_string,
logger_debug,
)

#############
Expand Down Expand Up @@ -80,7 +79,6 @@ class GraphDict(UserDict[str, Any]):
:type graph_name: str
"""

@logger_debug
def __init__(self, db: StandardDatabase, graph: Graph, *args: Any, **kwargs: Any):
super().__init__(*args, **kwargs)
self.data: dict[str, Any] = {}
Expand Down Expand Up @@ -111,7 +109,6 @@ def __process_graph_dict_value(self, key: str, value: Any) -> Any:
return graph_attr_dict

@key_is_string
@logger_debug
def __contains__(self, key: str) -> bool:
"""'foo' in G.graph"""
if key in self.data:
Expand All @@ -120,7 +117,6 @@ def __contains__(self, key: str) -> bool:
return aql_doc_has_key(self.db, self.graph_id, key)

@key_is_string
@logger_debug
def __getitem__(self, key: str) -> Any:
"""G.graph['foo']"""

Expand All @@ -139,7 +135,6 @@ def __getitem__(self, key: str) -> Any:

@key_is_string
@key_is_not_reserved
@logger_debug
def __setitem__(self, key: str, value: Any) -> None:
"""G.graph['foo'] = 'bar'"""
if value is None:
Expand All @@ -152,15 +147,13 @@ def __setitem__(self, key: str, value: Any) -> None:

@key_is_string
@key_is_not_reserved
@logger_debug
def __delitem__(self, key: str) -> None:
"""del G.graph['foo']"""
self.data.pop(key, None)
doc_update(self.db, self.graph_id, {key: None})

# @values_are_json_serializable # TODO?
@logger_debug
def update(self, attrs: Any) -> None:
def update(self, attrs: Any) -> None: # type: ignore
"""G.graph.update({'foo': 'bar'})"""

if not attrs:
Expand All @@ -173,7 +166,6 @@ def update(self, attrs: Any) -> None:
self.data.update(graph_attr_dict_data)
doc_update(self.db, self.graph_id, attrs)

@logger_debug
def clear(self) -> None:
"""G.graph.clear()"""
self.data.clear()
Expand All @@ -194,7 +186,6 @@ class GraphAttrDict(UserDict[str, Any]):
:type graph_id: str
"""

@logger_debug
def __init__(
self,
db: StandardDatabase,
Expand All @@ -219,7 +210,6 @@ def clear(self) -> None:
raise NotImplementedError("Cannot clear GraphAttrDict")

@key_is_string
@logger_debug
def __contains__(self, key: str) -> bool:
"""'bar' in G.graph['foo']"""
if key in self.data:
Expand All @@ -228,7 +218,6 @@ def __contains__(self, key: str) -> bool:
return aql_doc_has_key(self.db, self.graph.name, key, self.parent_keys)

@key_is_string
@logger_debug
def __getitem__(self, key: str) -> Any:
"""G.graph['foo']['bar']"""

Expand All @@ -246,7 +235,6 @@ def __getitem__(self, key: str) -> Any:
return graph_attr_dict_value

@key_is_string
@logger_debug
def __setitem__(self, key, value):
"""
G.graph['foo'] = 'bar'
Expand All @@ -263,15 +251,13 @@ def __setitem__(self, key, value):
doc_update(self.db, self.graph_id, update_dict)

@key_is_string
@logger_debug
def __delitem__(self, key):
"""del G.graph['foo']['bar']"""
self.data.pop(key, None)
update_dict = get_update_dict(self.parent_keys, {key: None})
doc_update(self.db, self.graph_id, update_dict)

@logger_debug
def update(self, attrs: Any) -> None:
def update(self, attrs: Any) -> None: # type: ignore
"""G.graph['foo'].update({'bar': 'baz'})"""
if not attrs:
return
Expand Down
29 changes: 7 additions & 22 deletions nx_arangodb/classes/dict/node.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@
key_is_string,
keys_are_not_reserved,
keys_are_strings,
logger_debug,
separate_nodes_by_collections,
upsert_collection_documents,
vertex_get,
Expand Down Expand Up @@ -97,7 +96,6 @@ class NodeAttrDict(UserDict[str, Any]):
:type graph: Graph
"""

@logger_debug
def __init__(self, db: StandardDatabase, graph: Graph, *args: Any, **kwargs: Any):
super().__init__(*args, **kwargs)
self.data: dict[str, Any] = {}
Expand All @@ -116,11 +114,9 @@ def clear(self) -> None:
raise NotImplementedError("Cannot clear NodeAttrDict")

def copy(self) -> Any:
# TODO: REVISIT THIS
return self.data.copy()

@key_is_string
@logger_debug
def __contains__(self, key: str) -> bool:
"""'foo' in G._node['node/1']"""
if key in self.data:
Expand All @@ -131,7 +127,6 @@ def __contains__(self, key: str) -> bool:
return result

@key_is_string
@logger_debug
def __getitem__(self, key: str) -> Any:
"""G._node['node/1']['foo']"""
if key in self.data:
Expand All @@ -151,7 +146,6 @@ def __getitem__(self, key: str) -> Any:
@key_is_string
@key_is_not_reserved
# @value_is_json_serializable # TODO?
@logger_debug
def __setitem__(self, key: str, value: Any) -> None:
"""
G._node['node/1']['foo'] = 'bar'
Expand All @@ -170,7 +164,6 @@ def __setitem__(self, key: str, value: Any) -> None:

@key_is_string
@key_is_not_reserved
@logger_debug
def __delitem__(self, key: str) -> None:
"""del G._node['node/1']['foo']"""
assert self.node_id
Expand All @@ -181,7 +174,6 @@ def __delitem__(self, key: str) -> None:
@keys_are_strings
@keys_are_not_reserved
# @values_are_json_serializable # TODO?
@logger_debug
def update(self, attrs: Any) -> None:
"""G._node['node/1'].update({'foo': 'bar'})"""
if not attrs:
Expand Down Expand Up @@ -213,7 +205,6 @@ class NodeDict(UserDict[str, NodeAttrDict]):
:type default_node_type: str
"""

@logger_debug
def __init__(
self,
db: StandardDatabase,
Expand Down Expand Up @@ -250,7 +241,6 @@ def __str__(self) -> str:
return self.__repr__()

@key_is_string
@logger_debug
def __contains__(self, key: str) -> bool:
"""'node/1' in G._node"""
node_id = get_node_id(key, self.default_node_type)
Expand All @@ -270,7 +260,6 @@ def __contains__(self, key: str) -> bool:
return False

@key_is_string
@logger_debug
def __getitem__(self, key: str) -> NodeAttrDict:
"""G._node['node/1']"""
node_id = get_node_id(key, self.default_node_type)
Expand All @@ -290,7 +279,6 @@ def __getitem__(self, key: str) -> NodeAttrDict:
raise KeyError(key)

@key_is_string
@logger_debug
def __setitem__(self, key: str, value: NodeAttrDict) -> None:
"""G._node['node/1'] = {'foo': 'bar'}

Expand All @@ -308,7 +296,6 @@ def __setitem__(self, key: str, value: NodeAttrDict) -> None:
self.data[node_id] = node_attr_dict

@key_is_string
@logger_debug
def __delitem__(self, key: str) -> None:
"""del g._node['node/1']"""
node_id = get_node_id(key, self.default_node_type)
Expand Down Expand Up @@ -336,7 +323,6 @@ def __delitem__(self, key: str) -> None:

self.data.pop(node_id, None)

@logger_debug
def __len__(self) -> int:
"""len(g._node)"""
return sum(
Expand All @@ -346,15 +332,13 @@ def __len__(self) -> int:
]
)

@logger_debug
def __iter__(self) -> Iterator[str]:
"""for k in g._node"""
if not (self.FETCHED_ALL_IDS or self.FETCHED_ALL_DATA):
self._fetch_all()

yield from self.data.keys()

@logger_debug
def keys(self) -> Any:
"""g._node.keys()"""
if self.FETCHED_ALL_IDS:
Expand All @@ -368,15 +352,20 @@ def keys(self) -> Any:
self.data[node_id] = empty_node_attr_dict
yield node_id

@logger_debug
def clear(self) -> None:
"""g._node.clear()"""
self.data.clear()
self.FETCHED_ALL_DATA = False
self.FETCHED_ALL_IDS = False

def copy(self) -> Any:
"""g._node.copy()"""
if not self.FETCHED_ALL_DATA:
self._fetch_all()

return {key: value.copy() for key, value in self.data.items()}

@keys_are_strings
@logger_debug
def __update_local_nodes(self, nodes: Any) -> None:
for node_id, node_data in nodes.items():
node_attr_dict = self.node_attr_dict_factory()
Expand All @@ -386,7 +375,6 @@ def __update_local_nodes(self, nodes: Any) -> None:
self.data[node_id] = node_attr_dict

@keys_are_strings
@logger_debug
def update(self, nodes: Any) -> None:
"""g._node.update({'node/1': {'foo': 'bar'}, 'node/2': {'baz': 'qux'}})"""
separated_by_collection = separate_nodes_by_collections(
Expand Down Expand Up @@ -414,15 +402,13 @@ def update(self, nodes: Any) -> None:
logger.warning(m)
raise ArangoDBBatchError(errors)

@logger_debug
def values(self) -> Any:
"""g._node.values()"""
if not self.FETCHED_ALL_DATA:
self._fetch_all()

yield from self.data.values()

@logger_debug
def items(self, data: str | None = None, default: Any | None = None) -> Any:
"""g._node.items() or G._node.items(data='foo')"""
if data is None:
Expand All @@ -434,7 +420,6 @@ def items(self, data: str | None = None, default: Any | None = None) -> Any:
v_cols = list(self.graph.vertex_collections())
yield from aql_fetch_data(self.db, v_cols, data, default)

@logger_debug
def _fetch_all(self):
self.clear()

Expand Down
14 changes: 14 additions & 0 deletions nx_arangodb/classes/digraph.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,14 @@ def __init__(
self.clear_edges = self.clear_edges_override
self.add_node = self.add_node_override
self.remove_node = self.remove_node_override
self.reverse = self.reverse_override

if (
not self.is_multigraph()
and incoming_graph_data is not None
and not self._loaded_incoming_graph_data
):
nx.convert.to_networkx_graph(incoming_graph_data, create_using=self)

#######################
# nx.DiGraph Overides #
Expand All @@ -79,6 +87,12 @@ def __init__(
# def out_edges(self):
# pass

def reverse_override(self, copy: bool = True) -> Any:
if copy is False:
raise NotImplementedError("In-place reverse is not supported yet.")

return super().reverse(copy=True)

def clear_edges_override(self):
logger.info("Note that clearing edges ony erases the edges in the local cache")
for predecessor_dict in self._pred.data.values():
Expand Down
28 changes: 10 additions & 18 deletions nx_arangodb/classes/function.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

from __future__ import annotations

from collections import UserDict
from typing import Any, Callable, Generator, Optional, Tuple

import networkx as nx
Expand Down Expand Up @@ -194,11 +195,11 @@ def wrapper(self: Any, key: Any, *args: Any, **kwargs: Any) -> Any:
""""""
if isinstance(key, str):
if key != "-1" and "/" not in key:
raise ValueError(f"{key} is not an ArangoDB ID.")
raise KeyError(f"{key} is not an ArangoDB ID.")

elif isinstance(key, int):
m = "Edge order is not guaranteed when using int as an edge key. It may raise a KeyError. Use at your own risk." # noqa
logger.warning(m)
logger.debug(m)

else:
raise TypeError(f"{key} is not an ArangoDB Edge _id or integer.")
Expand All @@ -208,16 +209,6 @@ def wrapper(self: Any, key: Any, *args: Any, **kwargs: Any) -> Any:
return wrapper


def logger_debug(func: Callable[..., Any]) -> Any:
"""Decorator to log debug messages."""

def wrapper(self: Any, *args: Any, **kwargs: Any) -> Any:
logger.debug(f"{type(self)}.{func.__name__} - {args} - {kwargs}")
return func(self, *args, **kwargs)

return wrapper


def keys_are_strings(func: Callable[..., Any]) -> Any:
"""Decorator to check if the keys are strings."""

Expand Down Expand Up @@ -447,10 +438,10 @@ def aql_edge_count_src(
direction: str,
) -> int:
query = f"""
RETURN LENGTH(
FOR v, e IN 1..1 {direction} @src_node_id GRAPH @graph_name
RETURN DISTINCT e._id
)
FOR v, e IN 1..1 {direction} @src_node_id GRAPH @graph_name
COLLECT id = e._id
COLLECT WITH COUNT INTO num
RETURN num
"""

bind_vars = {
Expand All @@ -475,8 +466,9 @@ def aql_edge_count_src_dst(
query = f"""
FOR v, e IN 1..1 {direction} @src_node_id GRAPH @graph_name
FILTER {filter_clause}
COLLECT WITH COUNT INTO length
RETURN length
COLLECT id = e._id
COLLECT WITH COUNT INTO num
RETURN num
"""

bind_vars = {
Expand Down
Loading