net: sockets: Implement sendto() and recvfrom() #1055

askawu · 2017-08-09T11:04:54Z

Support sendto() and recvfrom() for datagram socket.

tbursztyka · 2017-08-09T11:08:18Z

subsys/net/lib/sockets/sockets.c

add {} always, even on one liners. this is a zephyr style requirement

tbursztyka · 2017-08-09T11:11:58Z

subsys/net/lib/sockets/sockets.c

this is not a getter function anymore as you are appending data as well in the process, you'll have to rename this one.

tbursztyka · 2017-08-09T11:12:56Z

subsys/net/lib/sockets/sockets.c

reverse the order of all the 5 lines above.

tbursztyka · 2017-08-09T11:14:09Z

subsys/net/lib/sockets/sockets.c

lines break are cheap and help for code readability, add empty lines after variable declaration.

tbursztyka · 2017-08-09T11:14:42Z

subsys/net/ip/net_pkt.c

tbursztyka · 2017-08-09T11:16:23Z

subsys/net/ip/net_context.c

use IS_ENABLED with this CONFIG option, in the test right below.

Do the same for ipv4 also

pfalcon

As a quick reply, this seems more complex than intuitively expected.

My intuition can be wrong of course, but one issue I see is trying to resolve "default local bind for UDP" issue on BSD sockets level. This isn't good approach. We should resolve as many as possible issues on net_context (i.e. native Z API level), to make sure that it, and any levels above it benefit from it.

I introduced a framework for default local binding in #817, and applied it to net_context_connect(). It was on my TODO to apply it to UDP use cases which don't require explicit connect(), but that slipped.

So, I'd recommend to use the same approach and already introduced functions from #817 to handle unbound UDP context in net_context_send(to)() and net_context_recv().

Just as send/recv operations are usually on critical path, I'd suggest calling bind_default() only on UDP paths (as TCP should be already bound in connect()).

The above would be a separate PR. And actually, I'd suggest making inet_pton() fix (good catch!) its own PR too.

jukkar

I do not have anything extra, Tomasz pretty much commented everything already.

jukkar · 2017-08-09T12:42:48Z

I introduced a framework for default local binding in #817, and applied it to net_context_connect(). It was on my TODO to apply it to UDP use cases which don't require explicit connect(), but that slipped.

Indeed, good point.

askawu · 2017-08-10T10:03:19Z

@tbursztyka , I will correct the coding style problem. Thanks
@pfalcon , at the beginning, I tried to auto bind the local address by bind_default() in net context and it worked. But it become complicated when registering the recv callback before sendto(). Anyway, I will make the inet_pton patch as a new PR and see if I can make auto bind work for UDP in net context.

pfalcon · 2017-09-13T11:26:21Z

@askawu: Any updates here? Would you like me to give a try to resolving auto-bind issue for UDP sockets?

askawu · 2017-09-14T04:04:22Z

@pfalcon, I updated the patch but I forgot to update the PR. Can you help to check if a81cbe2 makes sense to you? bind_default() is used to auto bind UDP soccket.

pfalcon · 2017-09-21T11:08:59Z

subsys/net/ip/net_context.c

I would definitely suggest to call this only for UDP case, there's no need to do an extra call for TCP case. There're 2 ways to do that:

If we care about source addr to be bound for offloading case, then just add bind_default() block wrapped in #if defined(CONFIG_NET_UDP) after existing #if defined(CONFIG_NET_TCP) right below.

If we can cheat on offloading case, can reuse existing if (net_context_get_ip_proto(context) == IPPROTO_UDP) far below.

@jukkar : What do you think?

Sure, makes sense.

@jukkar: Thanks for the reply, but it's "either/or" type of question (either way 1 above, or way 2) ;-). Please suggest which way to go.

@GAnthony : Maybe you can suggest here something too.

I prefer the second one.

pfalcon · 2017-09-21T12:43:44Z

subsys/net/ip/net_pkt.c

What about returning the address len in *addrlen? If you didn't intend to do that, why is it passed by pointer?

Yes, the address size isn't necessary to be a pointer.

Hmm, I'd say "it's useful to return the actual addrlen". Because otherwise, we probably don't even need to pass it in - struct sockaddr can accommodate both IPv4 and IPv6 addresses.

It's used to prevent out of bound access. For example, the address is IPv6 but a sockaddr_in variable is passed in. I know it's rare but the caller might make a mistake. In such case, an error will be returned, so I probably don't need to return the actual addrlen.

Ok, I assume you considered various usecases for this function, so sounds good. (And we can change it later if found useful.)

pfalcon · 2017-09-21T12:45:06Z

include/net/net_pkt.h

Once semantics of *addrlen figure out (comment below), this docstring should describe what gets there on input, what on output.

pfalcon · 2017-09-21T12:49:12Z

subsys/net/lib/sockets/sockets.c

This doesn't appear to be correct.

pfalcon · 2017-09-21T12:52:12Z

subsys/net/lib/sockets/sockets.c

I would do it differently - zsock_send() should just call to zsock_sendto() with NULL dest address. Given that there's issue with length returned (previous comment), you might need to it like that.

pfalcon · 2017-09-21T12:54:40Z

subsys/net/lib/sockets/sockets.c

Why is this line here? (And not in zsock_send()? zsock_send() and zsock_sendto() should behave the same, and as noted previously, the way to ensure that is make one tail-call another).

My assumption is, for TCP case, zsock_connect() will be called first and callback will be registered at that time. So I only add it in zsock_sendto().

If we make zsock_send() and zsock_send() in adhoc way, any idea to prevent the registering the callback twice? Is there a proper way in net context to check if the callback has been registered?

I see. Yep, there're some non-obvious points which only come to attention when you dig deeper into things. (Which may hint that there should be a code comment about it!)

If we make zsock_send() and zsock_send() in adhoc way, any idea to prevent the registering the callback twice? Is there a proper way in net context to check if the callback has been registered?

Well, so you already register a callback here multiple times - each call to sendto() will lead to callback being registered. If that works, then testing for UDP and setting it in send() should also work.

But overall, that's a similar problem to bind_default(). I'd say, making send() to tail-cal to sendto() is more priority than avoiding duplicate setting if recv callback. Let's try to optimize it as we can now (i.e. set it in send*() only for UDP case), and I already accept that we'll need a much later pass over the socket code to see how we can optimize things.

pfalcon · 2017-09-21T12:55:28Z

subsys/net/lib/sockets/sockets.c

Yes, this is right. zsock_send() should be done like this.

pfalcon · 2017-09-21T13:14:33Z

subsys/net/lib/sockets/sockets.c

Why this conditionalizing, leading to code duplication? If we (may) need a packet header, let's keep it here, and consistently remove in zsock_recvfrom.

Initially, I prefer to keep the packet header for both cases. But I found zsock_recv_stream() needs to be modified a lot because packet header isn't dropped. I will modify this to prefer consistency.

Hmm, you may be right - and we can't return src addr for an arbitrary TCP recv anyway (only for recv() which starts to consume a new packet). Well, that's why it's useful to know why something was implemented this or that way, so having extra info in the commit messages or PR comments is helpful. And yeah, I'd appreciate your giving another thought to trying to avoid that code duplication - if it will lead to only more code being added, sure, it's ok to leave it like that.

pfalcon · 2017-09-21T13:24:06Z

@askawu: Thanks for refactors so far, and sorry for the delay with the review.

Well, unfortunately it's still not there IMHO, some issues need fixing, and some things can be redone in an easier/cleaner way. Per-commit comments:

net: pkt: Add net_pkt_get_src_addr()

I was surprised that such a function needs to be introduced. How did we do it before? I checked echo_server and yeah, it picks src addr (to be dst addr on reply) in an adhoc way. @jukkar , how do you deal with it in http server? Would the func above be useful there, would you refactor to use it?

net: context: Bind default address for UDP

My usual suggestion goes here: please split unrelated patches to a standalobe PRs to allow for quicker review cycle. I also suggest to optimize TCP case (for which we know that context is already bound). There's a pending question to @jukkar how to best do that.

net: sockets: Implement sendto() and recvfrom()

zsock_recvfrom() is implemented like it should be - by making zsock_recv() to dispatch to it. There's still a code duplication to optimize though.

But instead of doing the same for zsock_sendto() and zsock_send(), it's done in adhoc way, which in particular leads to a bug with return length. Please redo the same way as zsock_recvfrom()/zsock_recv(). Besides being easier and cleaner that way, that's also definitely a requirement for socket offloading, I hope @GAnthony can second on that.

Thanks!

askawu · 2017-09-22T10:08:37Z

@pfalcon, Thanks for the comments. I will think about how to make zsock_send() and zsock_sendto() easier.
@jukkar, Can you help to review 8296c76? I would like to know if there's a better way to get the src address from the packet.

jukkar · 2017-09-27T19:41:43Z

subsys/net/ip/net_pkt.c

The pkt might not have context specified so this is error prone.
Family can be get using net_pkt_family(pkt) instead. For protocol, we could get that information from IP header, or we could store it to net_pkt header when packet is received. We have been trying to avoid increasing the size of net_pkt so first option could be done here.

So depending on family, the UDP/TCP information can be get like this
proto = NET_IPV6_HDR(pkt)->nexthdr;
or
proto = NET_IPV4_HDR(pkt)->proto;

jukkar · 2017-09-27T19:47:06Z

subsys/net/ip/net_pkt.c

If you move this block before checking UDP/TCP, then you could add the proto resolving here.

jukkar · 2017-09-27T19:53:02Z

I was surprised that such a function needs to be introduced. How did we do it before? I checked echo_server and yeah, it picks src addr (to be dst addr on reply) in an adhoc way. @jukkar , how do you deal with it in http server? Would the func above be useful there, would you refactor to use it?

That function sounds useful to have, +1 for that.

jukkar · 2017-09-27T19:55:16Z

I also suggest to optimize TCP case (for which we know that context is already bound). There's a pending question to @jukkar how to best do that.

I must have missed this question, could you elaborate what was this question about?

Introduce net_pkt_get_src_addr() as a helper function to get the source address and port from the packet header. Signed-off-by: Aska Wu <[email protected]>

This patch makes net_context_sendto() work independently without calling net_context_connect() first. It will bind default address and port if necessary. Also, since receive callback should be provided before sending data in order to receive the response, bind default address and port to prevent providing an unbound address and port to net_conn_register(). Signed-off-by: Aska Wu <[email protected]>

sendto() and recvfrom() are often used with datagram socket. sendto() is based on net_context_sendto() and recvfrom() is based on zsock_recv() with parsing source address from the packet header. Signed-off-by: Aska Wu <[email protected]>

- Add test cases of ipv4/ipv6 sendto() and recvfrom() - Set the main stack size to 2048 to enable both ipv4 and ipv6 on qemu_x86. - Use net app for network setup. Signed-off-by: Aska Wu <[email protected]>

askawu · 2017-09-28T08:22:40Z

Update:

net_pkt_get_src_addr() should use the family and protocol from packet header instead of the value in net context.
Bind default address and port only for UDP
Refactor zsock_send() and zsock_sendto() to remove the duplicated code.

jukkar

LGTM

pfalcon

LGTM, thanks for all the refactoring!

@tbursztyka : Your comments also should have been addressed, can you please have a look?

askawu requested review from jukkar and tbursztyka as code owners August 9, 2017 11:04

askawu force-pushed the sendto_and_recvfrom branch from fa1c014 to 3d25f3a Compare August 9, 2017 11:12

tbursztyka requested changes Aug 9, 2017

View reviewed changes

pfalcon suggested changes Aug 9, 2017

View reviewed changes

jukkar reviewed Aug 9, 2017

View reviewed changes

jukkar added the net label Aug 10, 2017

askawu force-pushed the sendto_and_recvfrom branch from 3d25f3a to 5aaee19 Compare August 23, 2017 10:05

pfalcon reviewed Sep 21, 2017

View reviewed changes

subsys/net/lib/sockets/sockets.c Outdated

Copy link

Contributor

pfalcon Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't appear to be correct.

pfalcon reviewed Sep 21, 2017

View reviewed changes

subsys/net/lib/sockets/sockets.c Outdated

Copy link

Contributor

pfalcon Sep 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is right. zsock_send() should be done like this.

pfalcon reviewed Sep 21, 2017

View reviewed changes

nashif added area: Networking and removed net labels Sep 23, 2017

askawu force-pushed the sendto_and_recvfrom branch from 5aaee19 to 047224c Compare September 26, 2017 08:14

jukkar requested changes Sep 27, 2017

View reviewed changes

Aska Wu added 2 commits September 28, 2017 15:06

net: pkt: Add net_pkt_get_src_addr()

666efec

Introduce net_pkt_get_src_addr() as a helper function to get the source address and port from the packet header. Signed-off-by: Aska Wu <[email protected]>

Aska Wu added 2 commits September 28, 2017 15:07

net: sockets: Implement sendto() and recvfrom()

dbd3e1a

sendto() and recvfrom() are often used with datagram socket. sendto() is based on net_context_sendto() and recvfrom() is based on zsock_recv() with parsing source address from the packet header. Signed-off-by: Aska Wu <[email protected]>

tests: sockets: Add tests of sendto() and recvfrom()

d73c931

- Add test cases of ipv4/ipv6 sendto() and recvfrom() - Set the main stack size to 2048 to enable both ipv4 and ipv6 on qemu_x86. - Use net app for network setup. Signed-off-by: Aska Wu <[email protected]>

askawu force-pushed the sendto_and_recvfrom branch from 047224c to d73c931 Compare September 28, 2017 07:14

jukkar approved these changes Sep 28, 2017

View reviewed changes

pfalcon approved these changes Oct 3, 2017

View reviewed changes

tbursztyka approved these changes Oct 3, 2017

View reviewed changes

jukkar merged commit 7c60ead into zephyrproject-rtos:master Oct 3, 2017

pfalcon mentioned this pull request Oct 18, 2017

net: Duplicated functionality between net_pkt_get_src_addr() and net_context.c:create_sockaddr() #4421

Closed

net: sockets: Implement sendto() and recvfrom() #1055

net: sockets: Implement sendto() and recvfrom() #1055

Uh oh!

Conversation

askawu commented Aug 9, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pfalcon left a comment

Choose a reason for hiding this comment

Uh oh!

jukkar left a comment

Choose a reason for hiding this comment

Uh oh!

jukkar commented Aug 9, 2017

Uh oh!

askawu commented Aug 10, 2017

Uh oh!

pfalcon commented Sep 13, 2017

Uh oh!

askawu commented Sep 14, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pfalcon commented Sep 21, 2017

Uh oh!

askawu commented Sep 22, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jukkar commented Sep 27, 2017 •

edited

Loading