Today I enhanced debug mode and provided for a debug logfile. I also did a bunch of refactoring and bug fixes... maybe half of what Perl Critic suggested. This isn't an urgent release unless you're having trouble, might as well wait for 3.0.

http://www.droppedpackets.org/scripts/ldms_core/ldms_core_2-0.zip/view

I've got some reports of problems with NMAP... it crashes and takes ldms_core.exe with it. That's not supposed to happen, so if it's happening to you, please let me know and send a debug file.

UPDATE: Talk about relentless... Arnold Garcia tested with the new debug stuff and helped me find a problem, along with some more debugging suggestions. I did some more refactoring as well, so I've posted again and let's call it 2.9.8. This is worth upgrading to if you've had trouble with NMAP.

The TODO list:

1. TODO -- Test email button in the GUI

2. TODO -- Setup should install a LANDesk LOCALEXEC script to run it

3. TODO -- Provide visual feedback of activity stage, level

4. TODO -- Check scheduled tasks and policies

5. TODO -- plot non-RFC1918 addresses on a map

6. TODO -- keep old information and show a trend.

7. TODO -- purty Charts, http://search.cpan.org/src/CHARTGRP/Chart-2.4.1/README

8. TODO -- when you find duplicate IP addresses, wipe out the one that has the

9. older update date.

10. TODO -- If the database isn't busy, reindex it? Detecting a need to reindex

11. the database seems to be just as intrusive as actually reindexing the

12. database. More research required before this can be done safely.

I've done a few installs of ldms_core on servers where XDD or scheduled UDD has been running for a good long while in a 5,000 to 10,000 node environment, and I've found a bit of a performance issue. Eventually it all works itself out, but that first run takes a day or two to complete. Gradually it gets faster and faster, and within a week it works the way it does for a fresh LANDesk core. This post is not to announce a fix, just acknowledging that I see the problem and know what needs to be done.

This is what's happening: If NMAP's binary is found in the configured place, then ldms_core goes to the database and looks for targets with no OS Name, sorted by lastscantime desc for a LIFO scan order. So far so good, except a few months ago I realized that some NMAP scans finish in a few seconds, while others take several minutes. In order to get the most success up front, I first send a single ping packet to each target and see if it responds. If it answers ping, it goes to the front of the list, and if it doesn't, it goes to the back of the list.

That works really well when the list of targets is a few dozen machines, but when it's a few hundred machines the performance is terrible. As NMAP updates the OS Names, the target list becomes smaller, the pings go faster, and less time is taken. I'm not parallelizing any of this traffic, so a long time spent on step 1 means step 2 can cool its heels in the corner. This all fits with my design goal of "low and slow", under-the-radar information gathering, but no one can tell that it's doing anything -- there's an icon in the system tray and a couple of entries in event viewer, but no CPU, no network, no nothing.

I've considering batching targets into small groups, which I could then do in parallel, but decided it was too complex and wasteful of resources. So, the planned fix is a two parter:

1. First, I'm going to log and balloon tip what's happening so that the end user knows that it's okay to leave it alone... killing the program and starting it over just makes it take longer, and all the statistics that they installed it for happen at the end.

2. Second, I'm going to radically decrease the ping timeout -- if the node doesn't ICMP REPLY in a second or two, I know what I came for. Better still, if the machine responds to ping I'll do NMAP right away, but if it doesn't respond to ping I'll push it to the end of the line.

If you really care a lot, the source is installed with the binary, and this is the DoNMAP subroutine beginning at line 1119.

http://www.droppedpackets.org/process-manager/ldms_nmap_lpm/

No doubt about it, this is a cool trick. However, I've gotten a lot of calls and emails from people building solutions based on these ideas, and I feel the need to clarify a few things:

• NMAP isn't perfect. Network health, distance from the target, age of nmap version, type of device being scanned, and the weather in Katmandu can cause its accuracy to vary wildly.

• LANDesk isn't NMAP. If you're getting goofy results, your best (only?) recourse is to work with the NMAP team to improve the results: http://nmap.org/osdetect/osdetect-unidentified.html

• Even at less than 50% accuracy, the tool can still be useful. This trick is typically employed in environments where a network security breach is extremely important, or where the tracked equipment is worth as much as a new car. If an XDD event is not worth getting out of your chair to investigate, then integrating NMAP into LANDesk may not be worth your time... but if it is, using this sort of technique can be a lifesaver at any accuracy level.