Linode Forum

[bcoker]

Greetings Linoders,

I have a file named domains_list.txt with a list of domains.

Code: domain1.com
domain2.com
domain3.com

I need to run this command for each domain in domains_list.txt and echo it to a file named servers_list.txt.

nslookup domain1.com | grep -i "Server:" >> servers_list.txt

The output in the servers_list.txt should look something like this;

Code: domain1.com Server:   ip address
domain2.com Server:   ip address
domain3.com Server:   ip address

I can do all these things individually, but putting them all together is proving difficult.

Maybe someone already has something like this that I can modify or someone could get me started in the direction.

Thanks much!

[db3l]

You don't mention what shell you are using, but this will work with bash (should also work with sh and probably others). Put whatever command you want to execute per-domain inside the backticks. Output on stdout, so redirect wherever you want.

Code: cat domains_list.txt | while read domain; do echo $domain: `nslookup $domain | grep -i Server:`; done
Another option would be xargs:

Code: cat domains_list.txt | xargs -L 1 -I DOM echo DOM `nslookup DOM | grep Server:`
I guess I tend to prefer a shell loop when the original contents are interleaved in the output (as opposed to just being an argument for the command xargs is running), but that's just personal preference.

-- David

[bcoker]

David,

Thank you for your reply. I will take a look and test this shortly. It's been one of those Mondays. :) I'll report back my results.

[bcoker]

David,

Thanks again for the reply. Well it works to some degree. I needed to chagnged nslookup to dig. I'm not sure why I said nslookup. That should'nt change how the script works though.

If I run this from Command line;
Code:  - dig test.com | grep ns

I get the following;
Code: l options:  printcmd
;; Got answer:
domain.com.      60   IN   NS   ns2.dotsterhost.com.
domain.com.      60   IN   NS   ns3.dotsterhost.com.
domain.com.      60   IN   NS   ns1.dotsterhost.com.

This is good enough for me. However when I do it from command line or shell script using the command you gave me. I get the following;
Eample shell;
Code: #!/bin/sh
 cat domains.txt | xargs -L 1 -I DOM echo DOM `dig DOM | grep ns`

Example output;
Code: domain.com ;; global options: printcmd ;; Got answer: . 56315 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2011060601 1800 900 604800 86400
It's like the dig DOM is empty? Any ideas?

Thank you!

[db3l]

Not entirely sure. Something must be happening with the dig query when run beneath xargs - or somehow the string replacement is including hidden whitespace or something odd. When I try (but remove the grep part) I can see I'm getting back an NXDOMAIN error rather than a NOERROR like you do interactively, which then accounts for the difference in the output.

The shell loop works the same in either case, so I'd just use that instead. Maybe it would have been simpler not to mention the xargs case since it's not normally what I reach for in cases like this, though I did test it with nslookup and it seemed fine.

-- David

[bcoker]

David,

Sorry for the late reply, i've been super busy. I went with the first example and with minors changes got what I needed. Thanks for taking the time out to assist. It is very much appreciated.

Bill.

[shippinpie]

Hi bcoker, I believe this would be an easier way to accomplish what you want to:

Code: for i in `cat domains_list.txt`; do dig $i | grep ns >> output.txt; done

Basically, this is saying "for each domain in the domains file, dig the current domain, grep for NS, and append it to output.txt". Hope that makes sense?

[sweh]

shippinpie wrote: Code: for i in `cat domains_list.txt`; do dig $i | grep ns >> output.txt; done
I always find it funny to see people do greps and appends inside a loop like that.
Code: for i in `cat source_file`; do some_cmd $i; done | grep srch_string > output.txt is so much more efficient in terms of file open/close and in terms of processes being forked. You can even reduce it one more with a while-loop
Code: while read i; do some_cmd $i; done < source_file | grep srch_string > output.txt

Of course your original code doesn't meet the OPs requirements, but that's a different question entirely :-)
Code: while read i
do
  dig $i ns
done < domains_list.txt | grep -w NS | grep -v '^;' > output.txt
And with some optimisation:
Code: while read i
do
  dig $i ns
done < domains_list.txt | awk '!/^;/ && /\<NS\>/ {print $1,$5}' > output.txt

[shippinpie]

Admittedly, I only briefly read through the OP's requirements, and did the code based on what I saw. I appreciate what you've written out and shown me, and I'll be sure to look through this and optimize my shell scripts taking this into account in future. I appreciate the time you've taken - thanks :)

Edit:
While I found the first of your two codes worked:
Code: while read i
do
  dig $i ns
done < domains_list.txt | grep -w NS | grep -v '^;' > output.txt

The second of your two codes did not produce any results for me:
Code: while read i
do
  dig $i ns
done < domains_list.txt | awk '!/^;/ && /\<NS\>/ {print $1,$5}' > output.txt
Are others having similar issues, or is this simply my machine? The issue seems to be with the /\<NS\>/ section, so I'm thinking maybe my results just aren't being returned in a similar fashion to others...

[sweh]

The only difference between the 2 code variants is that I replaced the two grep's with an awk, and the awk also prints columns 1 and 5 rather than the whole line.

Thus something that looks like Code: example.com  86400 IN  NS name.server.example.com should output Code: example.com name.server.example.com
I had tested it on my CentOS 5 machine and it worked fine.

Thinking about it, though, we can somplify even more; the awk part could be Code: awk '$1 !~ /^;/ && $4 == "NS" { print $1,$5}'
And, of course, the dig output doesn't have $4=NS on the comment lines, so it's even simpler as Code: awk '$4 == "NS" { print $1,$5}'

This makes the whole loop: Code: while read i
do
  dig $i ns
done < domains_list.txt | awk '$4 == "NS" {print $1,$5}' > output.txt

Example results: Code: % cat domains_list.txt
google.com
facebook.com

% while read i
do
  dig $i ns
done < domains_list.txt | awk '$4 == "NS" {print $1,$5}'
google.com. ns1.google.com.
google.com. ns2.google.com.
google.com. ns3.google.com.
google.com. ns4.google.com.
facebook.com. ns3.facebook.com.
facebook.com. ns4.facebook.com.
facebook.com. ns5.facebook.com.
facebook.com. ns1.facebook.com.
facebook.com. ns2.facebook.com.

[shippinpie]

Cool - thanks for the input; I always enjoy learning new things ;)

[bcoker]

Guys,

Thank you for the additional follow ups. I and I'm sure many others that come across this will find it very helpful!

[Vance]

If you have version 9 or later of dig, you can eliminate the shell loop:
Code: $ dig -t NS -f domains_list.txt | awk '$4 == "NS" {print $1,$5}'
google.com. ns3.google.com.
google.com. ns4.google.com.
google.com. ns2.google.com.
google.com. ns1.google.com.
facebook.com. ns2.facebook.com.
facebook.com. ns3.facebook.com.
facebook.com. ns4.facebook.com.
facebook.com. ns5.facebook.com.
facebook.com. ns1.facebook.com.