Error 1 Sending The Modular Data For Mem_free
Sign in Pricing Blog Support Search GitHub This repository Watch 64 Star 326 Fork 194 ganglia/monitor-core Code Issues 51 Pull requests 7 Projects 0 Wiki Pulse Graphs New issue syslog fills with "Error 1 sending the modular data", gmond keeps using socket after EINVAL #65 Open dpocock opened this Issue Oct 28, 2012 · 2 comments Projects None yet Labels None yet Milestone No milestone Assignees No one assigned 3 participants Ganglia Development Team member dpocock commented Oct 28, 2012 Suggested action/solution: if write returns EINVAL, gmond should try to recreate or re-bind the sending socket, rather than continuing to send on a bad socket (and filling logs with errors) Google reveals this has been discussed several times in the past, and none of the discussions ended with a solution, so I'm presenting some analysis below. Here is what I did and what I found: I discovered my gmond PID = 21015 and I checked it with strace: strace -p 21015 -o /tmp/gmond.errs -v After about a minute, I had a look inside /tmp/gmond.errs, lots of this: write(7, "\0\0\0\205\0\0\0\4srv1\0\0\0\fmachine_type\0\0\0\0"..., 52) = 52 write(8, "\0\0\0\205\0\0\0\4srv1\0\0\0\fmachine_type\0\0\0\0"..., 52) = -1 EINVAL (Invalid argument) write(7, "\0\0\0\200\0\0\0\4srv1\0\0\0\7os_name\0\0\0\0\0\0\0\0\6"..., 164) = 164 write(8, "\0\0\0\200\0\0\0\4srv1\0\0\0\7os_name\0\0\0\0\0\0\0\0\6"..., 164) = -1 EINVAL (Invalid argument) time([1351418592]) = 1351418592 sendto(9, "<30>Oct 28 11:03:12 /usr/sbin/gm"..., 90, MSG_NOSIGNAL, NULL, 0) = 90 Notice the `sendto' is actually sending the error to syslog, not sending a metric packet Ok, the `write' calls show me two file descriptors, 7 and 8. writes to FD 8 are failing with EINVAL: write(8, .... ) = -1 EINVAL (Invalid argument) The file descriptors correspo
:""" /usr/sbin/gmond[4556]: Error 1 sending the modular data for """I see a hundred of them by second.Regards, Bernard Li 2011-05-03 18:23:23 UTC PermalinkRaw Message Does Anybody knows how i can delete/fix thousands of errors in syslog """ /usr/sbin/gmond[4556]: Error 1 sending the modular data for """ I see a hundred of them by second.Can you please provide additional info regarding your setup?- OS- Multicast or Unicast- Are you getting the error message for *all gmonds* or just some?- Are you getting the error for all metrics or just some? And if so,which ones?Can you https://github.com/ganglia/monitor-core/issues/65 please post your full gmond.conf somewhere likehttp://www.pastebin.com and reference it here?Thanks,Bernard Iban Cabrillo 2011-05-04 06:59:42 UTC PermalinkRaw Message Hi Bernard,- OSWe are using SL5 as SO with ganglia-gmond-3.1.7.- Multicast or Unicast- Are you getting the error message for *all gmonds* or just some?Yes, from *all gmonds* more than 200- Are you getting the error for all metrics or just some? And if so,which http://ganglia-general.narkive.com/l1lX5cbJ/error-1-sending-the-modular-data ones?Yes I think most of the metrics:/usr/sbin/gmond[4366]: Error 1 sending the modular data for mem_free/usr/sbin/gmond[4366]: Error 1 sending the modular data for cpu_user/usr/sbin/gmond[4366]: Error 1 sending the modular data for heartbeat/usr/sbin/gmond[4366]: Error 1 sending the modular data for mem_free/usr/sbin/gmond[4366]: Error 1 sending the modular data for pkts_in/usr/sbin/gmond[4366]: Error 1 sending the modular data for cpu_user/usr/sbin/gmond[4366]: Error 1 sending the modular data for heartbeat/usr/sbin/gmond[4366]: Error 1 sending the modular data for disk_free/usr/sbin/gmond[4366]: Error 1 sending the modular data for load_one/usr/sbin/gmond[4366]: Error 1 sending the modular data for cpu_user/usr/sbin/gmond[4366]: Error 1 sending the modular data for cpu_idle/usr/sbin/gmond[4366]: Error 1 sending the modular data for gexec/usr/sbin/gmond[4366]: Error 1 sending the modular data for proc_run/usr/sbin/gmond[4366]: Error 1 sending the modular data for mem_free/usr/sbin/gmond[4366]: Error 1 sending the modular data for mem_cached/usr/sbin/gmond[4366]: Error 1 sending the modular data for heartbeat/usr/sbin/gmond[4366]: Error 1 sending the modular data for cpu_user/usr/sbin/gmond[4366]: Error 1 sending the modular data for cpu_nice/usr/sbin/gmond[4366]: Error 1 sending the modular data for heartbeat/usr/sbin/gmond[4366]: Error 1 sending the modular data for mem_free/usr/sbin/gmond[4366]: Error 1 sending the modular data for heartbeat/usr/sbin/gmond[4366]: Error 1 sending the modular data for cpu_user/usr/sbin/gmond[4366]: Error 1 sending
() mail ! gmail ! com [Download message RAW] [Attachment #2 (multipart/alternative)] Hi Bernard, - OS We are using SL5 as SO with ganglia-gmond-3.1.7. - Multicast http://marc.info/?l=ganglia-general&m=131692063804097 or Unicast - Are you getting the error message for *all gmonds* or just some? Yes, from *all gmonds* more than 200 - Are you getting the error for all metrics https://jira.hpdd.intel.com/browse/LU-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&showAll=true or just some? And if so, which ones? Yes I think most of the metrics: /usr/sbin/gmond[4366]: Error 1 sending the modular data for mem_free /usr/sbin/gmond[4366]: Error 1 sending the modular data error 1 for cpu_user /usr/sbin/gmond[4366]: Error 1 sending the modular data for heartbeat /usr/sbin/gmond[4366]: Error 1 sending the modular data for mem_free /usr/sbin/gmond[4366]: Error 1 sending the modular data for pkts_in /usr/sbin/gmond[4366]: Error 1 sending the modular data for cpu_user /usr/sbin/gmond[4366]: Error 1 sending the modular data for heartbeat /usr/sbin/gmond[4366]: Error 1 sending the modular data for disk_free /usr/sbin/gmond[4366]: Error 1 sending the modular data error 1 sending for load_one /usr/sbin/gmond[4366]: Error 1 sending the modular data for cpu_user /usr/sbin/gmond[4366]: Error 1 sending the modular data for cpu_idle /usr/sbin/gmond[4366]: Error 1 sending the modular data for gexec /usr/sbin/gmond[4366]: Error 1 sending the modular data for proc_run /usr/sbin/gmond[4366]: Error 1 sending the modular data for mem_free /usr/sbin/gmond[4366]: Error 1 sending the modular data for mem_cached /usr/sbin/gmond[4366]: Error 1 sending the modular data for heartbeat /usr/sbin/gmond[4366]: Error 1 sending the modular data for cpu_user /usr/sbin/gmond[4366]: Error 1 sending the modular data for cpu_nice /usr/sbin/gmond[4366]: Error 1 sending the modular data for heartbeat /usr/sbin/gmond[4366]: Error 1 sending the modular data for mem_free /usr/sbin/gmond[4366]: Error 1 sending the modular data for heartbeat /usr/sbin/gmond[4366]: Error 1 sending the modular data for cpu_user /usr/sbin/gmond[4366]: Error 1 sending the modular data for heartbeat /usr/sbin/gmond[4366]: Error 1 sending the modular data for load_one /usr/sbin/gmond[4366]: Error 1 sending the modular data for proc_run /usr/sbin/gmond[4366]: Error 1 sending the modular data for mem_free /usr/sbin/gmond[4366]: Error 1 sending the modular data for bytes_out /usr/sbin/gmond[4366]: Error 1 sending the modular data for bytes_in /usr/sbin/gmond[4366]: Error 1 sending the modular data for cpu_user /usr/sbin/gmond[4366]: Error 1 s
start run: umount of OST failsAgile Board ExportXMLWordPrintable Details Type: Bug Status: Resolved Priority: Major Resolution: Fixed Affects Version/s: Lustre 2.4.0, Lustre 2.4.1, Lustre 2.5.0, Lustre 2.4.2, Lustre 2.5.1 Fix Version/s: Lustre 2.6.0, Lustre 2.5.1 Labels: mn4 zfs Severity: 3 Rank (Obsolete): 7893 Description This issue was created by maloo for Nathaniel Clark