Linode Forum
Linode Community Forums 		 FAQFAQ    SearchSearch    MembersMembers      Register Register 
 LoginLogin [ Anonymous ] 
Post new topic  Reply to topic

Board index » Linux Community Forums » Linux Tips, Tricks, Tutorials
  Previous topic | Next topic 
Author Message
bcoker
 Post subject: Shell Script Help
PostPosted: Mon Jun 06, 2011 10:31 am 
Offline
Senior Newbie

Joined: Tue Sep 07, 2010 12:19 pm
Posts: 14
AOL: 7000+Springs+LN
Location: United States
Greetings Linoders,

I have a file named domains_list.txt with a list of domains.

Code:
domain1.com
domain2.com
domain3.com


I need to run this command for each domain in domains_list.txt and echo it to a file named servers_list.txt.

nslookup domain1.com | grep -i "Server:" >> servers_list.txt

The output in the servers_list.txt should look something like this;

Code:
domain1.com Server:   ip address
domain2.com Server:   ip address
domain3.com Server:   ip address


I can do all these things individually, but putting them all together is proving difficult.

Maybe someone already has something like this that I can modify or someone could get me started in the direction.

Thanks much!
Top
   
Reply with quote  
db3l
 Post subject:
PostPosted: Mon Jun 06, 2011 2:09 pm 
Offline
Senior Member

Joined: Wed May 13, 2009 1:18 am
Posts: 681
You don't mention what shell you are using, but this will work with bash (should also work with sh and probably others). Put whatever command you want to execute per-domain inside the backticks. Output on stdout, so redirect wherever you want.

Code:
cat domains_list.txt | while read domain; do echo $domain: `nslookup $domain | grep -i Server:`; done

Another option would be xargs:

Code:
cat domains_list.txt | xargs -L 1 -I DOM echo DOM `nslookup DOM | grep Server:`

I guess I tend to prefer a shell loop when the original contents are interleaved in the output (as opposed to just being an argument for the command xargs is running), but that's just personal preference.

-- David
Top
   
Reply with quote  
bcoker
 Post subject:
PostPosted: Mon Jun 06, 2011 7:32 pm 
Offline
Senior Newbie

Joined: Tue Sep 07, 2010 12:19 pm
Posts: 14
AOL: 7000+Springs+LN
Location: United States
David,

Thank you for your reply. I will take a look and test this shortly. It's been one of those Mondays. :) I'll report back my results.
Top
   
Reply with quote  
bcoker
 Post subject:
PostPosted: Tue Jun 07, 2011 12:52 am 
Offline
Senior Newbie

Joined: Tue Sep 07, 2010 12:19 pm
Posts: 14
AOL: 7000+Springs+LN
Location: United States
David,

Thanks again for the reply. Well it works to some degree. I needed to chagnged nslookup to dig. I'm not sure why I said nslookup. That should'nt change how the script works though.

If I run this from Command line;
Code:
 - dig test.com | grep ns


I get the following;
Code:
l options:  printcmd
;; Got answer:
domain.com.      60   IN   NS   ns2.dotsterhost.com.
domain.com.      60   IN   NS   ns3.dotsterhost.com.
domain.com.      60   IN   NS   ns1.dotsterhost.com.

This is good enough for me. However when I do it from command line or shell script using the command you gave me. I get the following;
Eample shell;
Code:
#!/bin/sh
 cat domains.txt | xargs -L 1 -I DOM echo DOM `dig DOM | grep ns`

Example output;
Code:
domain.com ;; global options: printcmd ;; Got answer: . 56315 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2011060601 1800 900 604800 86400

It's like the dig DOM is empty? Any ideas?

Thank you!
Top
   
Reply with quote  
db3l
 Post subject:
PostPosted: Tue Jun 07, 2011 2:12 am 
Offline
Senior Member

Joined: Wed May 13, 2009 1:18 am
Posts: 681
Not entirely sure. Something must be happening with the dig query when run beneath xargs - or somehow the string replacement is including hidden whitespace or something odd. When I try (but remove the grep part) I can see I'm getting back an NXDOMAIN error rather than a NOERROR like you do interactively, which then accounts for the difference in the output.

The shell loop works the same in either case, so I'd just use that instead. Maybe it would have been simpler not to mention the xargs case since it's not normally what I reach for in cases like this, though I did test it with nslookup and it seemed fine.

-- David
Top
   
Reply with quote  
bcoker
 Post subject:
PostPosted: Wed Jun 15, 2011 10:45 am 
Offline
Senior Newbie

Joined: Tue Sep 07, 2010 12:19 pm
Posts: 14
AOL: 7000+Springs+LN
Location: United States
David,

Sorry for the late reply, i've been super busy. I went with the first example and with minors changes got what I needed. Thanks for taking the time out to assist. It is very much appreciated.

Bill.
Top
   
Reply with quote  
shippinpie
 Post subject: Simpler way
PostPosted: Sun Jul 03, 2011 9:55 pm 
Offline
Senior Newbie

Joined: Thu Jun 24, 2010 10:07 am
Posts: 18
Hi bcoker, I believe this would be an easier way to accomplish what you want to:

Code:
for i in `cat domains_list.txt`; do dig $i | grep ns >> output.txt; done


Basically, this is saying "for each domain in the domains file, dig the current domain, grep for NS, and append it to output.txt". Hope that makes sense?
Top
   
Reply with quote  
sweh
 Post subject: Re: Simpler way
PostPosted: Sun Jul 03, 2011 11:06 pm 
Offline
Senior Member
User avatar

Joined: Tue Apr 13, 2004 6:54 pm
Posts: 833
shippinpie wrote:
Code:
for i in `cat domains_list.txt`; do dig $i | grep ns >> output.txt; done

I always find it funny to see people do greps and appends inside a loop like that.
Code:
for i in `cat source_file`; do some_cmd $i; done | grep srch_string > output.txt
is so much more efficient in terms of file open/close and in terms of processes being forked. You can even reduce it one more with a while-loop
Code:
while read i; do some_cmd $i; done < source_file | grep srch_string > output.txt


Of course your original code doesn't meet the OPs requirements, but that's a different question entirely :-)
Code:
while read i
do
  dig $i ns
done < domains_list.txt | grep -w NS | grep -v '^;' > output.txt

And with some optimisation:
Code:
while read i
do
  dig $i ns
done < domains_list.txt | awk '!/^;/ && /\<NS\>/ {print $1,$5}' > output.txt

_________________
Rgds
Stephen
(Linux user since kernel version 0.11)
Top
   
Reply with quote  
shippinpie
 Post subject: Re: Simpler way
PostPosted: Mon Jul 04, 2011 12:25 am 
Offline
Senior Newbie

Joined: Thu Jun 24, 2010 10:07 am
Posts: 18
Admittedly, I only briefly read through the OP's requirements, and did the code based on what I saw. I appreciate what you've written out and shown me, and I'll be sure to look through this and optimize my shell scripts taking this into account in future. I appreciate the time you've taken - thanks :)

Edit:
While I found the first of your two codes worked:
Code:
while read i
do
  dig $i ns
done < domains_list.txt | grep -w NS | grep -v '^;' > output.txt


The second of your two codes did not produce any results for me:
Code:
while read i
do
  dig $i ns
done < domains_list.txt | awk '!/^;/ && /\<NS\>/ {print $1,$5}' > output.txt

Are others having similar issues, or is this simply my machine? The issue seems to be with the /\<NS\>/ section, so I'm thinking maybe my results just aren't being returned in a similar fashion to others...
Top
   
Reply with quote  
sweh
 Post subject:
PostPosted: Mon Jul 04, 2011 11:30 am 
Offline
Senior Member
User avatar

Joined: Tue Apr 13, 2004 6:54 pm
Posts: 833
The only difference between the 2 code variants is that I replaced the two grep's with an awk, and the awk also prints columns 1 and 5 rather than the whole line.

Thus something that looks like
Code:
example.com  86400 IN  NS name.server.example.com
should output
Code:
example.com name.server.example.com

I had tested it on my CentOS 5 machine and it worked fine.

Thinking about it, though, we can somplify even more; the awk part could be
Code:
awk '$1 !~ /^;/ && $4 == "NS" { print $1,$5}'

And, of course, the dig output doesn't have $4=NS on the comment lines, so it's even simpler as
Code:
awk '$4 == "NS" { print $1,$5}'


This makes the whole loop:
Code:
while read i
do
  dig $i ns
done < domains_list.txt | awk '$4 == "NS" {print $1,$5}' > output.txt


Example results:
Code:
% cat domains_list.txt 
google.com
facebook.com

% while read i
do
  dig $i ns
done < domains_list.txt | awk '$4 == "NS" {print $1,$5}'
google.com. ns1.google.com.
google.com. ns2.google.com.
google.com. ns3.google.com.
google.com. ns4.google.com.
facebook.com. ns3.facebook.com.
facebook.com. ns4.facebook.com.
facebook.com. ns5.facebook.com.
facebook.com. ns1.facebook.com.
facebook.com. ns2.facebook.com.

_________________
Rgds

Stephen

(Linux user since kernel version 0.11)
Top
   
Reply with quote  
shippinpie
 Post subject:
PostPosted: Mon Jul 04, 2011 8:06 pm 
Offline
Senior Newbie

Joined: Thu Jun 24, 2010 10:07 am
Posts: 18
Cool - thanks for the input; I always enjoy learning new things ;)
Top
   
Reply with quote  
bcoker
 Post subject:
PostPosted: Tue Jul 05, 2011 4:27 pm 
Offline
Senior Newbie

Joined: Tue Sep 07, 2010 12:19 pm
Posts: 14
AOL: 7000+Springs+LN
Location: United States
Guys,

Thank you for the additional follow ups. I and I'm sure many others that come across this will find it very helpful!
Top
   
Reply with quote  
Vance
 Post subject:
PostPosted: Sat Jul 09, 2011 11:44 am 
Offline
Senior Member
User avatar

Joined: Sun Jan 18, 2009 2:41 pm
Posts: 830
If you have version 9 or later of dig, you can eliminate the shell loop:
Code:
$ dig -t NS -f domains_list.txt | awk '$4 == "NS" {print $1,$5}'
google.com. ns3.google.com.
google.com. ns4.google.com.
google.com. ns2.google.com.
google.com. ns1.google.com.
facebook.com. ns2.facebook.com.
facebook.com. ns3.facebook.com.
facebook.com. ns4.facebook.com.
facebook.com. ns5.facebook.com.
facebook.com. ns1.facebook.com.
Top
   
Reply with quote  
Display posts from previous:  Sort by  
Post new topic  Reply to topic

Board index » Linux Community Forums » Linux Tips, Tricks, Tutorials


Who is online

Users browsing this forum: No registered users and 1 guest

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
RSS

Powered by phpBB® Forum Software © phpBB Group