1 2 Previous Next

Jack Coates' Blog

30 Posts tagged with the droppedpackets tag

log monitoring tool

Posted by Jack Coates Jul 14, 2009

Here's a useful little tool that aggregates the various logfiles that LANDesk produces: http://www.droppedpackets.org/scripts/ldms_log

 

You'll find core and client programs, no installer necessary.

ldms_log_core.png

0 Comments Permalink

ldms_client 2.4.9 hits the web today. This is a major revision, because it moves the inventory locations of several attributes, and it provides an optional ability to model data for those attributes. The affected inventory items are Battery, NetStat, Mapped Drives, and User Profile Size.

 

What does that mean? From the manual:

A database schema is the map of the database; it's how programs know where to read and write data from. LANDesk's database has an extremely clever trick available to it, which allows it to extend that map on the fly. If you send it data that it doesn't know how to map, it will figure out what you wanted and make it happen. For instance, if you add "Foo"."Bar" = "Baz" to an inventory scan, it will reason that you must want to assign Baz to a value named Bar, which is under an object named Foo. So it takes the data, puts together a attribute relationship, and stores it all in the UNMODELEDDATA tables.

 

By default, much of the data that ldms_client provides is unmodeled... after all, if it were modeled that would mean that LDMS was already gathering it, and ldms_client wouldn't need to. If you just need ldms_client temporarily, this is fine. For example, if you're just trying to quickly ascertain license compliance or force Dell WoL into a known state, you probably don't want to make a permanent change to your database.

 

On the other hand, performance of unmodeled data is potentially very poor, especially as the number of nodes reporting it goes up. That means that if you're using some of ldms_client's features as a permanent part of your inventory, you'll want to model the data.

Cropper-1-06-02-2009-11-44-17 AM.Png

This is still optional because it's potentially risky, but the benefits are worth pursuing.

0 Comments Permalink

http://www.droppedpackets.org/management-gateway-and-remote-control/ldms_auto_gateway for details, but at version 1.0.8 it is successfully deployed at several customers of greater than 1,000 nodes.

14 Comments Permalink

Thanks to some questions from Jim Hysell, I found a bug in ldms_core yesterday, which has been fixed in 3.6.4. The bug was in the listing of detected vulns and autofix vulns by severity, which are then added together to make the detected and autofix lines in the Vulnerabiltiy Statistics graph. Prior to 3.6.4, the number is only accurate if you haven't created overlapping scopes. Of course, since everything's covered by the default "All machines" scope, that effectively means that all scopes overlap at least little. This caused the vulns on the machines that are in overlap to be counted once per scope that the machine was in, which is probably not what anyone wants.

 

Net result: For some LDSS admins, upgrading to ldms_core 3.6.4 or greater will cause a precipitous drop in the number of detected and autofix vulnerabilities. The new number is more accurate than the older one.

ldssstats-daily.png
So, in figuring that out I also realized that I could delineate between vulns detected, and machines with vulns, which wasn't very clear in the text report. There's a big difference between "you've got six critical vulns in the environment" and "you've got six machines with critical vulns on them"... These values are now clarified:

 

Detected vulnerability counts by severity:
Critical - 68 vulnerabilities found on 8 machines.
High - 45 vulnerabilities found on 7 machines.
Low - 12 vulnerabilities found on 9 machines.
Medium - 48 vulnerabilities found on 10 machines.
N/A - 129 vulnerabilities found on 11 machines.
Service Pack - 10 vulnerabilities found on 3 machines.
Unknown - 11 vulnerabilities found on 9 machines.


Detected vulnerabilities set to autofix by severity:
Critical - 64 vulnerabilities found on 7 machines.
High - 40 vulnerabilities found on 5 machines.
Low - 4 vulnerabilities found on 4 machines.
Medium - 12 vulnerabilities found on 7 machines.
N/A - 47 vulnerabilities found on 7 machines.
Service Pack - 1 vulnerability found on 1 machine.
Unknown - 5 vulnerabilities found on 7 machines.

 

By the way, I've also posted ldms_core to google code, so anyone so inclined can use the source code more easily. I'll do the same for my other projects as time permits.

0 Comments Permalink

I just posted ldms_core 3.4.7, which integrates some of the feedback and discoveries of the last few weeks... thanks for all the help, folks. Debugging help and feature suggestions from LANDesk admins around the world are making this utility into a very useful tool indeed.

tall-tree.jpg

Here's a few notes about where it might (or might not) go next:

  • Scheduled Tasks
  •   
       
    •  
    •  
    • Check scheduled tasks and policies for RRD stats -- jobs without start times, jobs in success-level buckets, duplicated jobs...   
          
    •  
    •  
    •  
    • Delete ghost devices from scheduled tasks (stuck in active because they reported status). If they were from a query they should be deleted from the list, but if they were from a static targeting they should be moved to pending. http://community.landesk.com/support/message/17222#17222   
          
    •  
    •  
  • Import   ldms_delete_users, auto-reassign to single user or delete objects, give the user an option to decide what should be done. Alternatively, rewrite ldms_delete_users as a standalone tool...
  • NMAP as an XDD client add-on instead of a core-side piece... this implies some command-channel use and data-passing which are non-trivial, but entirely possible. On the plus side, it will also produce a much higher level of accuracy in OS fingerprinting.
  • Email
  •   
       
    •  
    •  
    • Be smart about hysteresis... maybe it could not send another email within a day unless the new email it wants to send is more urgent than the last email that it had to send? Users going from daily runs to hourly runs are having challenges sorting the important emails from the repetitive info.
    •  
    •  
    •  
    • Maybe it's email worthy that unmanaged nodes isn't fresh...   
          
    •  
    •  
  • Web pages and reports
  •   
       
    •  
    •  
    • In RRD pages, give textual data supporting the graph. That'll probably push it over the edge to needing templated data instead of straight html.   
          
    •  
    •  
    •  
    • Support proxy servers (nice to have for update check, will need for geo-location)   
          
    •  
    •  
    •  
    • Give links to non-RFC1918 addresses on maps: GeoIP2Location   
          
    •  
    •  
    •  
    • Drill-down from topology map with per subnet listings of computers, including inventory and remote control links for them   
          
    •  
    •  
  • Auto-import email from domain controller into ConsoleUser table. If UserName is like Directory and Email is blank, then import from AD. Requires AD credential input in UI.
  • Count duplicate serial number records and show a count before the number... e.g. "34 machines with serial number SystemSerialNumb, 2 machines with LYAC12"
  • More options, more smarts, more feedback, more efficiency...
  • Find why McAfee silently stops it from working properly when it's run as a scheduled task (error 0x9 in Windows scheduled task, immediate "success" as a LANDesk scheduled task, works great when run interactively from the start menu).
2 Comments Permalink

ldms_core 3.3.3

Posted by Jack Coates Jan 6, 2009

Finally got the network map looking like I wanted it. You'll want to delete \\CORE\ldreports\ldms_core\ldms_core.css and replace it with the one from ldms_core 3.3.3 to enjoy this mofessional lookin style. Or don't, if you like that really ugly green on black look. I'd really love to see how some bigger maps get laid out, if someone could post some screenshots that would be great.

Cropper-1-01-06-2009-01-41-43 PM.Png

2 Comments Permalink

Any member may post articles anywhere, within the guidelines set on the main page.

 

Member accounts without associated email addresses have been deleted... you may be affected if you haven't used it in the last few months.

0 Comments Permalink

Version 3.2.6 is now uploaded to http://www.droppedpackets.org/scripts/ldms_core.

 

As the next step in its ongoing quest to delete data that was automatically gathered and then left to rot, ldms_core will now delete automatically gathered SLM products that have no installations (note that this is not USAGE, but INSTALLS).

 

It is possible that this can produce churn in situations where products are being installed and uninstalled in rapid succession; this should be a minor problem compared to the database performance degradation that results from thousands of junk product definitions.

 

As is usual with new features, this needs testing; please give it a whirl on your test server before trying it on production. At the very least, have a database backup handy.

3 Comments Permalink

ldms_client 1.7

Posted by Jack Coates Nov 21, 2008

Aside from the usual bugfixes and cleanup, this release adds configurable control over how hard ldms_client will look for email folders, and a report of the machine's FQDN.

 

Code and instructions can be found here: http://www.droppedpackets.org/inventory-and-slm/ldms_client

 

The conversations around future release ideas for ldms_core have proven interesting and useful, so here's an attempt to gather some of the loose notes:

  • TODO -- Firewall status is enabled /disabled. We do this in custom defs, but don’t put the status anywhere.
  • TODO -- It would be interesting to see the firewall specifics broken down a bit by port and app, but that could be a ton of work and difficult to format in a useful way.
  • TODO -- Capture the connected SSID. Something Perlish? WMI? netsh? All have problems.
  • TODO -- Custom vulnerability to install/update Produkey? Is there a better way to use Produkey, or do the same thing without it?
  • TODO -- Find a way to identify Adobe licenses... even if it's not decrypted, an identifiable GUID would be useful.
  • TODO -- Report system crashes from event log.
  • TODO -- The ongoing search for greater efficiency.
13 Comments Permalink

ldms_core 3.2.0

Posted by Jack Coates Nov 12, 2008

service checking now looks at the Avocent Management Platform services, too. I've also added code to look for schema faults. Ever see messages in your event viewer that say "column whatever is too small, increase it by so much?"

 

0 Comments Permalink

There'll be rejoicing in the alleyways tonight! http://www.droppedpackets.org/

If you'd like to see the gory details, read on.

Firstly, I wouldn't feel comfortable telling customers to test thoroughly and carefully if I wasn't personally inclined toward shooting from the hip. Backups were taken, but this upgrade was not really tested meaningfully. That's why it took about 15 hours of work instead of 5.

  • Hardware -- It's more fun to compute like it's 1999. Don't go changing!
  • OS Upgrade -- smooth and well-played. Moving from the execrable OpenSuSE 10.0 to Xubuntu 8.04 was a good idea.
  • Zope/Plone Upgrade -- and this is where the trouble begins. To use the package manager, or to use source? The Ubuntu packages for Zope and Plone are less than perfect; they're old, they're goofily designed, and they're a bit buggy. But, the install from source method is difficult and unpleasant to maintain... so when it didn't work and consumed a bit of troubleshooting time, I went back to the packages. That settled things on Zope 2.3 > Zope 2.9 and Plone 2.1.1 > Plone 2.5.1, and older revisions of some other packages that I wanted upgraded (such as Kupu, which is now at 1.3.9).
  • Site Restoration -- You simply restore the Data.fs file and restart, right? Well, it references products which may or may not exist, and it also does a bunch of magic which may or may not work. I must have wiped the machine back to a zope-free state and started over 6 or 7 times. I can tell you from memory that line 81 of the post-install for Ubuntu's plone-site package has a typo, for instance (setuser zope:zope, not setuser zope).
  • Site Upgrade -- You simply go to product_migration in the ZMI and click migrate. Except you also need to do that on portal_atct (undocumented), and if you do it without deleting your cookie_authentication object you'll be locked out of the site, and the dry-run option doesn't really dry run, and don't forget to delete your customizations from the old school days because they'll make things break. It wasn't as bad as installing Oracle, but it wasn't fun.

There was more, and there's still more to do, but it'll wait. I'm outta here for the weekend.

duct tape.jpg

0 Comments Permalink

ldms_core 3.0.1

Posted by Jack Coates Sep 29, 2008

At something like 90 to 360 seconds per scan, it makes no sense to NMAP thousands of nodes in a single run; target lists are now capped at 100 nodes per run, 50 if debug is on. Fixed a couple of bugs, too.

 

The manual is getting pretty out of date, I need to do something about that.

 

download here.

4 Comments Permalink

no-longer beta ldms_core

Posted by Jack Coates Aug 11, 2008

I've posted a beta release of ldms_core at http://www.droppedpackets.org/scripts/ldms_core/ldms_core_2-9-5beta.zip which attempts to support email authentication. If you'd like to try it out, please let me know of any errors.

 

17 days without a complaint, it must be working. 2.9.5 is published.

6 Comments Permalink

One of the more powerful tools that LANDesk gives you is a local scheduler with all sorts of useful state filters and activity triggers... it's the Local Scheduler that you're configuring when you select inventory, policy, and vulnerability scan windows in Agent Configuration, and it's able to do some powerful things: http://community.landesk.com/support/docs/DOC-2328

 

 

One of its most useful features is the /IPADDR trigger, which causes a script to run at IP Address changes. By default this is used to launch inventory scans, but it's also useful for making decisions that depend on which netwok you've just joined... this is the concept that drives auto_gateway.vbs.

 

 

The problem with auto_gateway.vbs is that you want it to go onto systems that are, well, using the gateway... in other words, connectivity is spotty, it might not be able to download from internal servers, and you want something that will keep trying until it gets it right. Something like a vulnerability scan, in other words. Here's a custom vulnerability that installs auto_gateway and defines a Local Scheduler task to run it -- a lot can be done to improve on it, but it's a good start for solving other network-based problems as well: http://www.droppedpackets.org/security/custom-definitions/v_intl_auto_gateway-installation.xml/view

 

 

The next amazing trick is to make the vulnerability only apply to laptops... a dummy product install would do that nicely, but there ought to be a cleaner way.

0 Comments Permalink

ldms_status bug fix

Posted by Jack Coates Aug 2, 2008

http://www.droppedpackets.org/scripts/ldms_status

 

I was at a customer Friday, and discovered that when I hovered over ldms_status, it would turn red, flail madly, and keel over dead, sort of like Bowker holding a ball at first and trying to decide what to do with it. Turns out, their server had 1 Inventory Service thread and 4500 clients, so LDSCAN contained a backlog of 4300 inventory scans, and growing. Whenever CountPendingScans ran, ldms_status was correctly deciding to restart the inventory service. It was incorrectly ignoring several instructions to wait a few seconds, and it was incorrectly doing this over and over as long as I hovered over the icon, causing a flickering stream of balloon tips and event viewer messages and doing nothing for the backlog of scans.

 

Partially, this is a discovery of unexpected behavior from the poorly documented perltray... I'm discovering that the Tooltip subroutine runs repeatedly as long as you're hovering on the icon, and I suspect that it sets the Timer too, overriding my timer settings. But the other part is that I was sloppy about the tooltip in the first place, and was calling code on demand. So, I decoupled the CountPendingScans subroutine from ToolTip. I'd been meaning to do that anyway as a matter of good practice, so finding a bug is just good impetus to do it right.

0 Comments Permalink
1 2 Previous Next
LANDesk Community powered by Jive Software's Clearspace ® Subscribe| Legal Notices| Investor Relations| Avocent| Privacy Policy © 2009 LANDesk Software