udns questions

[Michael Tokarev]

Markus Koetter wrote:
> Michael Tokarev wrote:
[]
> k, then this was an apu misunderstanding on my side,
> as thought of dns_context * beeing the context for one request, so
> int dns_getsock(dns_context *, int fd)
> should return the single socket a request can have.
> i dont like this
> int dns_getsocks(dns_context *, int fd[], int fdcnt)
> as its very hard to use in an loop/application

And this is very important misunderstanding, which leads to
"bogus" conclusions and suggestions (in this your reply too).

There are two main object types used in the library, both
opaque to an application but with some methods to access them.
They are:

   dns_context -- this is main resolver object that holds
    "resolver configuration" (timeouts, search lists etc),
    all sockets, list of nameservers, and an "active requests
    queue".  This is the object you use most of the time to
    submit queries, obtain active sockets, check for timeouts
    etc.

  dns_query -- this is a query object, or request object --
    single DNS request attached to a dns_context (in its
    active queue).  This dns_query object holds request-specific
    application data (callback and user pointer).

So there are already two objects - query-specific (dns_query)
and "global" dns_context.

> i'm used to non blocking tcp io, so maybe i can help you out
> getsockopt() is the magic word dealing with non blocking tcp sockets

TCP sockets aren't an issue, UDP ones are.

>> Note again: there will be either simple but not-for-everything API, or
>> very generic but difficult to use.  Dunno whenever you noticied that,
>> but some rr-specific routines does not accept "flags" argument, while
>> other does.  Eg, the one asking for PTR accepts no flags, while the
>> one asking for A does.  This is because the only flag of interest here
>> is whenever the DN being requested is "fqdn" or not, ie, whenever to
>> perform default-domain-suffix search for the request or not.  Ofcourse
>> there's no point to perform search for 1.2.3.4.in-addr.arpa. name.
>>
>> Now imagine each query-submitting routine has another parameter,
>> timeout (and, number of retries, and (which is quite useful sometimes!)
>> even a list of nameservers!) -- how complex the whole stuff will be!
>>
>> Ofcourse it is possible to modify some query-specific stuff after it
>> has been submitted.  I may even delay actual query dispatching till
>> the next event loop iteration (currently newly submitted query is sent
>> right when it is submitted), to allow setting some per-query options.
>> Well, now that's an idea really -- this way I can even achive some
>> code cleanups!.. ;)
> 
> i dont want you to rewrite it before its released, but maybe its worth a 
> try splitting dns_context into
> dns_context -- context for a single request
> and
> udns_context -- context -keeping many dns_requests- for many requests
> ?

It is already split this way -- see above.

> - considering the context splitting as 'multiple contexts' one could say 
> it offers simple way to make sure the right sockets gets polled all the 
> time without user callbacks, as udns just sets the latest used socket 
> into the dns_context->socket

Even for a single request, there will be multiple sockets still.
If we have more than one nameserver to try for example, AND we
will use one UDP socket per a nameserver.

Right now udns works just fine with only one socket for everything,
however.

Note you don't have to implement your own queue processing for
active DNS requests - it is quite.. problematic, this is exactly
what udns library is supposed to do.  There are multiple nameservers,
retries, timeouts and stuff like that - quite alot of processing
for an average application.

[]
> as i guess it will take some time till ipv6 nameservers get a standard, 
> this is nothing i would complain about yet.
> getting ipv6 addresses as a request result seems much more important to me.

Ah.  Well, udns is designed to support different record types
and be easily extendable in this regard.  And yes, it supports
AAAA right now.

>> Speaking of which (others).  I *still* (after several years of
>> experiments in various applications!) can't understand how to work
>> properly with connected udp sockets.  The problem is that socket
>> operation may return an error related to *previous* operation,
>> and there should be a way to tell which op *really* failed.
> 
> wow, thats really new to me, but i guess this problem can be beaten if 
> you split the contexts and use a single socket per request, as you

Heh.  The problem happens only when using connected UDP sockets,
this has nothing to do with per-request socket "splitting".  It is
either we use one socket for all nameservers (unconnected) -- does
not matter if that's per-request or per-global-context, -- or
single connected socket for each nameserver.  The problem occurs
only with per-nameserver sockets.

[]
>>> - ugly struct i had to get my results from, been a pain to find out 
>>> how to get the ips by rr domains
>>
>>
> 
>> Are udns structs any better?  I found them pretty ugly too... ;)
>> Especially now when I'm thinking about adding some decoders for
>> some more request types, with processing of additional sections --
>> like, to request MX records *together* with all A/AAAA records
>> necessary to perform actual mail delivery, or to request *and
>> verify* a PTR record - get PTR, get A/AAAA and compare with
>> original A/AAAA -- the structs will be quite.. ugly I think.
> 
> short adns example
> 
> int i;
> struct in_addr *mysi_addr = answer->rrs.inaddr;
> for ( i=0;  answer->nrrs > i; i++ )
> {
>     LogSpam("Resolved #%i %s \n",
>      i, inet_ntoa((struct in_addr) mysi_addr[i] ));
> }

Quite a strange typecast i'd say..  It should be
   inet_ntoa(*((struct in_addr*)&mysi_addr[i]))
instead, I think.

> does not look that hard, but it was hard to findout which value carries 
> what, i used insight & gdb to get the correct names.

Lack of documentation, really.  BTW, improving udns manpage is
in my todo list as well... ;)

> as i think splitting the dns_context into
> dns_context -- per request
> and
> udns_context -- for many requests
> is a really good idea, i will note some pro arguments here.
> 
> - multiple ways to use non blocking tcp sockets
>   - the callback per dns_context you set up as an idea
>   - setting the socket in the dns_struct so the poll has to poll the
>     correct socket in the next loop when he builds up the new pollfd set
>     i dunno if this works with kqueue or epoll
>   - getsockets(udns_context *,  int fd[], int numfd) could be used too,
>     even if i dont like it

Why you don't like it?  Remember, it's not per request, it's per-context
which is for many requests, you get the sockets once at initialisation
time and set them up in your poolfd/fd_set once and forall.  I mean
the current *static* socket(s) (there's only one right now, but may
be more in the future if i'll switch to per-nameserver sockets).

Ok, this looks even better:

  const int *dns_getsocks(const struct dns_ctx *ctx) --

this returns pointer to an internal array, terminated with -1,
with all the active sockets -- you don't have to allocate
fd[] array.  And ofcourse there will be a constant, like
DNS_MAXSOCKETS (pretty small value, currently 7 at max),
which will be sufficient to hold all sockets of a context
at any given time, too.

> - proper errorhandling, as you got one active socket per connection its
>   easy to say what went wrong on which dns_context request
>   - possibilty to recognize dead or bad nameservers faster
> - one api to use a single dns_context, one api to use the udns_context,
>   the udns_context api can use dns_context api as lower layer
> - best possible timeouthandling
>   - per dns_context
>   - or per udns_context
> - udns_context has internal polling io to be used in blocking or non
>   blocking mode, dns_context lacks this for obvious reasons
>   either poll it on your own, or use a udns_context
> - allows complete non blocking io with proper errorhandling for udp and
>   tcp sockets
> - even the send on a tcp socket can be non blocking as the context can
>   tell which events should be queued POLLIN|POLLOUT, dns_context buffers
>   the query untill its sended completly
> - follows the old roman "divide et impera" - "partition and command"
>   principle

Oh well.  I can't comment on most of the above, because it all comes from
the wrong conclusion about contexts vs queries.

I want to avoid usage of multiple contexts in applications except when
really necessary (I already provided two examples where multiple contexts
can be used -- special-purpose applications like dns debugging tools,
and multi-threaded apps with per-thread contexts).  Instead, everything
is about requests (dns_query) which are attached to a single context.

Yes, having per-request sockets is possible, and unavoidable for TCP
mode (in which case that socket can still be per-context, with a constraint
that only one tcp connection (per-context again) can be active at any
given time).

/mjt