Problem with publication information by BDII-Local

[jas01]

I got a alert on

midmon.egi.eu

on my bdii about information the bdii publish

CRITICAL - errors 78, warnings 26, info 52
Summary per type of error, warning and info message:
E022 - Default value published (GLUE2ComputingShareWaitingJobs): 26
E023 - Default value published (GLUE2ComputingShareEstimatedAverageWaitingTime): 26
E024 - Default value published (GLUE2ComputingShareEstimatedWorstWaitingTime): 26
I046 - Number of seconds higher than 1 million (GLUE2ComputingShareEstimatedAverageWaitingTime): 26
I047 - Number of seconds higher than 1 million (GLUE2ComputingShareEstimatedWorstWaitingTime): 26
W025 - Incoherent number of total jobs (GLUE2ComputingShareTotalJobs): 26

[Pansanel]

Which is the version of glite-info-provider-service?

[jas01]

Actually on my bdii-local I've

glite-info-provider-service-1.13.4-1.el6.noarch

Regards

[Pansanel]

Can you try to downgrade to 1.13.3-1 to see if it works better?

[Pansanel]

Dismiss the previous comment. I have found where the code is:
/usr/libexec/info-dynamic-pbs
Which version of lcg-info-dynamic-scheduler-pbs are you using?

[jas01]

The downgrade doen't change anything. But you already known that ;-)

I've split my torque server and the cream-ce.

On both

lcg-info-dynamic-scheduler-pbs-2.4.5-1.el6.noarch

regards.

[jrha]

Is this a problem with the Quattor template library or a middleware problem?

[Pansanel]

It may be related to Quattor if it is a gip parser configuration problem. It is not sure.
We have the same rpm (lcg-info-dynamic-scheduler-pbs-2.4.5-1.el6.noarch).
Can you check the content of /var/lib/bdii/gip/plugin/lcg-info-dynamic-ce and try to execute it manually?

[jas01]

so in the /var/lib/bdii/gip/plugin/lcg-info-dynamic-ce I've got a very simple

[root@torque-grid ~]# more /var/lib/bdii/gip/plugin/lcg-info-dynamic-ce
#!/bin/sh
cat /swareas/dteam/gip-ce-dynamic-info-maui.cache
[root@torque-grid ~]#

and inside this /swareas/dteam/gip-ce-dynamic-info-maui.cache
I've got lot of information, If I stay on one vo

dn: GlueCEUniqueID=cream-ce-grid.obspm.fr:8443/cream-pbs-glast.org,Mds-Vo-name=resource,o=grid
GlueCEInfoLRMSVersion: 2.5.13
GlueCEInfoTotalCPUs: 112
GlueCEPolicyAssignedJobSlots: 112
GlueCEStateFreeCPUs: 0
GlueCEStateFreeJobSlots: 0
GlueCEPolicyMaxRunningJobs: 110
GlueCEPolicyMaxWallClockTime: 4320
GlueCEPolicyMaxCPUTime: 1440
GlueCEStateStatus: Production

dn: GLUE2ShareID=GLAST-ORG_GLAST-ORG_torque-grid.obspm.fr_ComputingElement,GLUE2ServiceID=torque-grid.obspm.fr_ComputingElement,GLUE2GroupID=resource,o=glue
GLUE2ComputingShareFreeSlots: 0
GLUE2ComputingShareMaxRunningJobs: 110
GLUE2ComputingShareMaxWaitingJobs: 220
GLUE2ComputingShareMaxTotalJobs: 330
GLUE2ComputingShareServingState: Production
GLUE2EntityCreationTime: 2015-03-18T14:55:02Z
GLUE2ComputingShareMaxWallTime: 4320
GLUE2ComputingShareDefaultWallTime: 4320
GLUE2ComputingShareMaxCPUTime: 1440
GLUE2ComputingShareDefaultCPUTime: 1440
GLUE2ComputingShareMaxSlotsPerJob: 1
GLUE2ComputingShareMaxMainMemory: 2000
GLUE2ComputingShareMaxVirtualMemory: 20000

but if I try to find some from the nagios alerte it's doent' exist like

[root@torque-grid ~]# grep GLUE2ComputingShareEstimatedAverageWaitingTime /swareas/dteam/gip-ce-dynamic-info-maui.cache
[root@torque-grid ~]#grep GLUE2ComputingShareEstimatedWorstWaitingTime /swareas/dteam/gip-ce-dynamic-info-maui.cache
[root@torque-grid ~]#

[Pansanel]

Which version of template-library-grid are you using? Many work has been done on glue2 static publishing in November. Are you sure you are up to date?

[jouvin]

We started with an email exchange with @jas01 and he confirmed me that he was using 14.10.0. As you said, it is not something we see in other sites so probably due to some local conditions but I guess this is coming from some wrong Quattor configuration at site.

We are running the same version of glite-info-provider-service as @jas01 so I'll try to run the GLUE validator on our config again to check that we don't have the same problem...

[jouvin]

I have checked and cannot reproduce the problem on our CE... which doesn't mean that there is no problem! I'm trying to think about what could help to troubleshoot the problem...

[jouvin]

From what I can see running glue-validator (*), I have the feeling that something is not working with the GIP cache. This cache is made of 2 files: /swareas/dteam/gip-ce-dynamic-info-maui.cache and /swareas/dteam/gip-ce-dynamic-info-scheduler.cache. The problematic information is coming from the latter one (gip-ce-dynamic-info-scheduler.cache). Could you check that this file is created and that you have not error in /var/spool/maui/gip-info-dynamic-scheduler-plugin.sh when running the dynamic-info-scheduler plugin.

(*) glue-validator -H topbdii.grif.fr -p 2170 -b GLUE2DomainID=OBSPM,GLUE2GroupID=grid,o=glue -k -v 3 egi-profile

[jas01]

Sorry I didn't see your email because it's tag as spam.

From what I can see running glue-validator (*), I have the feeling that
something is not working with the GIP cache. This cache is made of 2 files: /
swareas/dteam/gip-ce-dynamic-info-maui.cache and /swareas/dteam/
gip-ce-dynamic-info-scheduler.cache. The problematic information is coming from
the latter one (gip-ce-dynamic-info-scheduler.cache). Could you check that this
file is created and that you have not error in /var/spool/maui/
gip-info-dynamic-scheduler-plugin.sh when running the dynamic-info-scheduler
plugin.

Your right that is the problem. I don't have the symbolic link between

/var/lib/bdii/gip/ldif/static-file-all-CE-pbs.ldif

and

/var/glite/static-file-all-CE-pbs.ldif

I create that symlink manually and every thing seem correct now.

I don't understand how that link disapeare. I try to launch ncm-ncd and he
didn't create this link. Should I modify my own quattor template so it's
create this link ? (for next time I re-install the server)

Regards

[pigay]

Dear all,

We came to the same error in midmon (https://midmon.egi.eu/nagios/cgi-bin/extinfo.cgi?host=sitebdii.m3pec.u-bordeaux1.fr&type=2&service=org.bdii.GLUE2-Validate)

We run the same version of lcg-info-dynamic-scheduler-pbs as @jas01 also on 14.10 templates, but we can't find an error running /var/lib/bdii/gip/plugin/lcg-info-dynamic-ce or /var/spool/maui/ gip-info-dynamic-scheduler-plugin.sh.

Can you help us?

pierre