RadiSys ATCA-4616 Dokumentacja Strona 1

ReferenceSHELF MANAGEMENT SOFTWARESOFTWARE VERSION 4.2.xApril 2012 007-03261-0008

1Chapter10Shelf Management OverviewTheShelfManagerisresponsibleformonitoringconditionsofmodulesandothershelfcomponentsandcontrollingth

6Troubleshooting100General troubleshooting proceduresThissectionexplainshowtoacknowledgeandviewalarms,viewsystemevents,andchecksensoran

6Troubleshooting101Viewing the system event logYoucanviewamodule’slocalSELortheshelf’sSELusingtheopen‐sourceipmitoolutilityorthersys

6Troubleshooting102Investigating sensor valuesToviewthesensorvalueforasensorthathasgeneratedanalarm:1. WritedowntheFRUslotidentified

6Troubleshooting1032. ListthesensorsfortheSPMresource:qsb<resourceId>3. Notetheintaketemperatures.4. Repeatsteps1,2,and3fortheo

6Troubleshooting104On the ATCA-6014 and ATCA-6016 shelves:Eachfantrayhasanexhausttemperaturesensor.Checktheexhausttemperaturesasfollows:1

6Troubleshooting105IPMI frame structure and message flowTherearenoframesinIPMBmessages.Figure 10showsthemessageflowthroughanSCMpairfor

6Troubleshooting106OnlytheswitchingportionoftheSCMsactinActive‐Activemode.TheShMCsareneverindualactivestateunlessthere'sac

6Troubleshooting107EachIPMCthatisnottheactiveShMCmonitorsits“ownside”ofthebus.Inotherwords,eachIPMCismonitoringitsownI2Cinter

6Troubleshooting108TheusercandotoseveralthingsrespondtoapersistentIPMBisolationissuewhenthereisashelfwideIPMBisolationalarmthat

6Troubleshooting109d. ToreturntheboardtoitsdefaultstatewhereitisjoinedonthatIPMByoucanagainusetheCLIasfollows:platform‐mgmt(fron

1Shelf Management Overview11Event and error managementTheShelfManagermanagesnormaleventsanderrorconditionsintheseways:•Powersdevicesupan

6Troubleshooting110Troubleshooting FRU and shelf operation issuesThissectionexplainshowtorespondtoFRUandshelfoperationalarmsandotherissu

6Troubleshooting111Responding to specific alarmsResponding to a communication lost alarmTheShelfManagerisnotgettingkeep‐aliveresponsesfromthe

6Troubleshooting112TherearenohardwarepresencedetectlinesinATCAbackplanes.TheShMSthusreliesonmessagebasedpings toperiodicallyquery

6Troubleshooting113Aftermakinganyshmgr.conffileupdate,remembertosynchronizeittothestandbySCMfromtheplatform‐managementCLIbeforerebo

6Troubleshooting114Responding to an IPMB isolated alarmThemodule’sIPMChasdetectedaproblemononeorbothofitsIPMBinterfaces,andhasisolate

6Troubleshooting1155. Disableeventgenera tiononthatsensorwiththesecommands:8<resourceId><sensorId>Thescreendisplaysthefinalme

6Troubleshooting1165. Lookforthe“inetaddr”value.TheSCMinthelower‐numberedslotinthechassisshouldhaveaneth1IPaddressof10.0.1.1.Th

6Troubleshooting117Beforeaddressingthepossiblecauseandtryingtocorrectit,someoftheinformationfromtheINSERTION_PENDINGstatemaybeusef

6Troubleshooting1183. IftheFRUisfoundtobeinM2andAutoInsertiontimerisnot‐1oralargevalue,checkthe stateofthe'FRUActivation

6Troubleshooting119c. Checkthe'DesiredPower'controlsonallFRUsinstalledonthatslot(parentandmanagedFRUs)andcheckifthetotal

1Shelf Management Overview12Shelf Manager complianceAsofthismanual’spublicationdate,theRadisysShelfManagerisfullycompliantwiththePICMG

6Troubleshooting120Responding to a fan not running1. CheckthattheShelfManagercancommunicatewiththefanandthatitisintheactivestate.Use

6Troubleshooting121Troubleshooting Shelf Manager issuesGeneral issuesWhenyouencounterproblemsrelatedtoshelfmanagement,checkthefollowingitem

6Troubleshooting122Make sure the Shelf Manager software is runningMakesuretheShelfManagersoftwareisrunning.TheActiveLEDoftheSCM(oroneS

6Troubleshooting123Blade and platform management issuesThetopicsinthissectionincludeShMC‐to‐IPMCcommunications,SELlocation,sensoreventrepo

6Troubleshooting124TypicallyeachIPMChasonlyitsowneventsinitsSEL.TheShMCsareexceptionstothisrulebecausetheylogeventsfromallIPMC

6Troubleshooting125Inventory Data Record (IDR) topicsRadisyscanprovideatoolcalledfruupdatethatcanbeusedtoupda teexistingFRUinventoryda

6Troubleshooting126Specific issuesTable 15listsspecificissuesthatcanbeencounteredwhileusingtheshelfmanagementandHPIservicesontheShel

6Troubleshooting127(continued from previous page)If the modules are still in debug mode, use these steps:• From each Shelf Manager module, verify that

6Troubleshooting128SCM resets abruptly forcing the LMP to reboot. There are no associated hot-swap events. Possible causes and steps to take are as fo

6Troubleshooting129Sessions cannot be opened on the HPI server. Error code: SA_ERR_HPI_OUT_OF_SPACEThe HPI server has reached the limit (32) for the n

1Shelf Management Overview13Figure 3. Shelf management component distribution in larger Radisys shelvesCarrier moduleProxy FRUIPMCOther FRUPEMIPMCIPMB

6Troubleshooting130RPT is incomplete; not all FRUs present in the shelf have associated Resources.1. Initiate a discovery operation using the CLI comm

6Troubleshooting131Resolving shelf FRU information issuesThissectionprovidesinformationonresolvingshelfFRUinformationproblems,includinghow

6Troubleshooting132Restoring shelf FRU informationTheprocedurebelowisusefulwhentheshelfFRUinformationisbadandneedstobereplacedinthe

6Troubleshooting133Restoring shelf FRU information on ATCA-6002 shelvesTheATCA‐6002shelfusesafiletostoreshelfFRUinformation,becausetherei

6Troubleshooting134IPMB addresses of slots and FRUsTable 16showstheIPMBaddressesofslotsintheATCA‐6002shelf.Table 17showstheIPMBaddresse

6Troubleshooting135Resource types of slots and FRUs in the RPTTheresourcepresencetableidentifiesthetypeofeachresourceinthe2ndbyteofthe

6Troubleshooting136Thefollowingaretheslotresourcetypesinhex:F0 UnknownslotF1 SPMorRCMslotF2 ShelfdisplaypanelslotF3 VirtualSPMorvirt

AAppendix137Shelf Manager InitializationShelf Manager and HPI interface initializationTheshelfmanagementserver(ShMS)isinstalledaspartofthel

AShelf Manager Initialization138d. PopulatestheRPTwithresourcesandtheirassociatedresourcedatarecords(RDRs).Resourcesarecreatedforeach

AShelf Manager Initialization139HPI subagent initializationWhentheHPIsubagentisstarted,initiallyyouwillseejustthestaticscalarobjectsdef

1Shelf Management Overview14Major elements of the shelf management infrastructureThefollowingsectionsdescribethemajorelementsoftheshelfmanag

BAppendix140Shelf Management FilesThisappendixdescribesfilesintheshelfmanagementinfrastructure,includingprograms,scripts,andconfiguration

BShelf Management Files141Shelf Manager log fileTheShelfManagerlogfileis/tmp/shmgr/shmgr.log.Backupfilesinthesamedirectoryarecrea tedas

CAppendix142IPMI Commands and Managed SensorsSupported IPMI commandsTheShelfManagersupportsallIPMIcommandslistedintheAdvancedTCABaseSpecif

CIPMI Commands and Managed Sensors143Table 18. Managed sensors for the Shelf Manager# Name Type CategoryNormal reading or event stateExplanation of

CIPMI Commands and Managed Sensors144Table 19. Managed sensors for each FRU# Name Type CategoryNormal reading or event stateExplanation of values256(0

CIPMI Commands and Managed Sensors14530 Get Address Info failed31 Self Test Error32 Self Test Check failed33 Set IPMB-<A|B> State <Enable|Dis

CIPMI Commands and Managed Sensors146258(0x102)FRU Thermal StatusTemperature Threshold 0x1 State 0x1: Lower minor.State 0x2: Lower major.State 0x4: L

1Shelf Management Overview15Field replaceable units (FRUs)AdvancedTCAdefinestwotypesofFRUsthatarevisibletoandcontrolledthroughtheIPMIin

1Shelf Management Overview16Management controllers (IPMCs and MMCs)EachintelligentFRUincludesamanagementcontroller,eitheranIntelligentPlatfo

1Shelf Management Overview17Theshelfcontainsanon‐volatilestoragedevicethatstoresshelfFRUinformation,includingdetailsoftheshelfitself

2Chapter18Software ArchitectureShelf management architectureShelfmanagementservicesfortheRadisysplatformarepartitionedacrosstwoprocessors:

2Software Architecture19Shelf interfaceAttheshelfinterfacelevel,theShMCfirmwarerunsontheIPMC,performingthelow‐levelshelfmanagementope

Revision historyVersion Date Description-0000 December 2006 First edition.-0001 September 2007 Second edition. New features and editorial corrections

2Software Architecture20HPI architectureTheprimarymanagementinterfacetotheRadisysshelfmanagementservicesistheHardwarePlatformInterface(

2Software Architecture21AsshowninFigure 5,theHPIclientlibrary(HCL)isadynamicallylinkedlibrary(DLL)thatprovidestheHPIAPIaswellas

2Software Architecture22RMCP,arequest‐responseprotocol,isasimplepacket‐basedcommunicationmechanism.RMCPpack etsincludeafieldthatindica

2Software Architecture23Forinformationonviewingthelogmessages,seeViewingthesystemeventlogonpage 101.Eventsmaybeclearedfromtheshel

2Software Architecture24Basic and enhanced shelf management operationTheShelfManagerinanATCA‐6002shelfcanoperateintwomodes:enhancedandba

2Software Architecture25Inenhancedmode,theShelfManagerusesthe/etc/shmgr.conffile.TheShelfManagerdoesnotneedtousetheshmgr_basic.conf

2Software Architecture26Figure 7. SCM Shelf Manager redundancy Shelf Manager failover ComponentfailuresintheShMSandShMCaredetectedthroughthe

2Software Architecture27Peer communication loss causes failoverIfcommunicationbetweenthepeerShMCsbreaksdown,thefirstofthepreviouscommunic

2Software Architecture28•UsingthefrontpanelresetbuttontoresettheactiveSCMeithermanuallyorusingHPIwiththesaHPIResourceResetStateSet()

2Software Architecture29Shelf FRU information redundancyDuringnormalshelfoperationwithredundantShelfManagers,therearealwaysatleastthreep

3Table of ContentsPreface ...

2Software Architecture30Shelf FRU invalid alarmThisalarmsignalsacatastrophicfaultcondition.TheactiveShelfManagerkeepsallothermodulesin

2Software Architecture31HPI server redundancyTheRadisy sHPIserverusesaredundancymodelverysimilartotheactive‐standbymodelusedbytheShel

2Software Architecture32InadditiontotheHPIdata,automaticoperationsaresynchronizedwiththepeerHPIserverstokeepHPIdatabasesinsync.Th

2Software Architecture33Shelf coolingToensurethatallmodulesintheshelfoperateefficiently,theiroperationaltemperaturesmustbemaintainedat

2Software Architecture34IfbothSCMsareremovedfromtheATCA‐6014orATCA‐6016shelf,theactiveRCMadjuststhefanstofullspeeduntiltheShelf

2Software Architecture35Shelf Manager cooling overrideTheShelfManagerprovidesautonomouscoolingmanagementfortheshelf.However,theremaybesi

3Chapter36Configuring Shelf Manager and HPI BehaviorThischapterexplainshowto:• DisableorenabletheRadisysShelfManager.• Changetheoperating

3Configuring Shelf Manager and HPI Behavior37Disabling the Shelf ManagerImportant:WhenthereisonlyoneSCMorShelfManagermoduleintheshelf,di

3Configuring Shelf Manager and HPI Behavior38Enabling the Shelf ManagerImportant:Ifyouareremovingamodulefromanon‐RadisysorRadisysMPCH0001

3Configuring Shelf Manager and HPI Behavior39Changing the Shelf Manager operating modeTheShelfManagerinanATCA‐6002shelfcanoperateintwomodes

4Table of ContentsChapter 4: Changing Shelf Settings Using HPI ... 53Radisys

3Configuring Shelf Manager and HPI Behavior40Overview of Shelf Manager and HPI configurationShelfManagerandHPIconfigurationsettingscanbeeither

3Configuring Shelf Manager and HPI Behavior41Table 4. Shelf Manager and HPI configuration settingsShelf Manager configuration settings Parameter to ed

3Configuring Shelf Manager and HPI Behavior42Editing the shmgr.conf configuration file File‐basedconfigurationsettingsarestoredinthe/etc/shmgr.

3Configuring Shelf Manager and HPI Behavior43Configuring Shelf Manager behaviorThissectiondescribestheshelfmanagementconfigurationsettingsyou

3Configuring Shelf Manager and HPI Behavior44Disabling the HPI serviceTheSHMS_CMDLINE_ARGSparameterin/etc/shmgr.confletsyoudisableHPIservice.

3Configuring Shelf Manager and HPI Behavior45Configuring the system event log (SEL)TheSHMS_SYNC_SEL_TIMEparameterin/etc/shmgr.confsetsthe syste

3Configuring Shelf Manager and HPI Behavior46Configuring HPI behaviorThissectiondescribestheHPIconfigurationsettingsyoucanchangebyeditingH

3Configuring Shelf Manager and HPI Behavior47• HSD_CHASSIS_FRU_MONITOR_ENABLEenablesthemonitoringofshelfFRUs,includingthealarmmoduleandfan

3Configuring Shelf Manager and HPI Behavior48Assigning an entity type for a new FRUTheHSD_FRU_ENTITY_TYPEparameterin/etc/shmgr.confletsyouassig

3Configuring Shelf Manager and HPI Behavior49Loading saved HPI resource configuration filesTheHSD_LOAD_SAVED_CONFIGURATIONparameterin/etc/shmgr.co

5Table of ContentsAppendix A: Shelf Manager Initialization... 137She

3Configuring Shelf Manager and HPI Behavior50Configuring the Shelf Manager in basic modeForaShelfManagerrunningontheATCA‐6002shelf,aconfigur

3Configuring Shelf Manager and HPI Behavior51Editing the rsyshsd.conf configuration fileForeachfrontmodulethereareblade HPIconfigurationsetti

3Configuring Shelf Manager and HPI Behavior52Configuring log settings•TheRSYSHSD_LOG_FILEparameteriswhereyoucanspecifythenameofthelogfile

4Chapter53Changing Shelf Settings Using HPIThischapterdescribestheRadisysHPIimplementation,includinghowHPIaffectsyourapplications,howto

4Changing Shelf Settings Using HPI54TheDomainIdneededforopeningHPIsessionsontheRadisysplatform’sHPIInterfaceisderivedfromtheShelfMan

4Changing Shelf Settings Using HPI55ThefollowingistheResourceIdformatusedbyshelfandShelfManagerresourcesintheHPIresourcepresencetabl

4Changing Shelf Settings Using HPI56Using the platform-management CLIYoucanaccessseveralcommandmodes,includingplatform‐managementmode,fromth

4Changing Shelf Settings Using HPI57Writing HPI applicationsThefollowingarerequiredtocreateanHPIapplication:SaHpi.h TheofficialHPIB.03.02h

4Changing Shelf Settings Using HPI58TorunanyHPIapplicationthatlinkswiththeRadisysHPIclientlibrary,libhcl.so,considerthefollowing:•The

4Changing Shelf Settings Using HPI59Using OpenHPI toolsOpenHPIisanopensourceprojectcreate dwiththeintentofprovidinganimplementationofSA

6PrefaceAbout this manualThismanualdescribestheRadisysShelfManagementSoftware(the“ShelfManager”),theHardwarePlatformInterface(HPI)serv

4Changing Shelf Settings Using HPI60Using SNMPSNMPaccesstotheshelfmanagementfunctionalityisprovidedthroughtheHPI‐B0101‐MIBmodule,whichwo

4Changing Shelf Settings Using HPI61Eachofthesetablesisindexedby:saHpiDomainId,saHpiResourceIdsaHpiResourceIsHistoricalsaHpiCtrl*EntryIdobject

4Changing Shelf Settings Using HPI62Configuring HPI subagent loggingHPIsubagentloggingisdisabledbydefault.Toenablelogging:1. Inthe/etc/rc.d

4Changing Shelf Settings Using HPI63Using the example HPI application (hpiapp)RadisysprovidesanexampleHPIapplication,hpiapp,thatiscompliantw

4Changing Shelf Settings Using HPI64Running hpiappThefollowingexamplesshowhowtorunthehpiappexampleapplicationtoconnecttoanHPIserverru

4Changing Shelf Settings Using HPI65Changing the shelf address Thefirst4bytesoftheshelfaddressshouldbechangedmanuallytouniquelyidentify

4Changing Shelf Settings Using HPI66Changing the Shelf Manager IP address TheactiveshelfmanagementserverhasadedicatedIPaddressthatisderive

4Changing Shelf Settings Using HPI67YoucaneitherchangeonelineatatimeorchangeallthreelinestogetherusingthesaHpiControlSet()function.

4Changing Shelf Settings Using HPI68Changing the FRU power-on sequence using the CLIYoucanusetheCLIto:•Viewthecurrentpower‐onsequence• Chang

4Changing Shelf Settings Using HPI69FRU power-on sequence controlsTheshelfresource(ResourceId0x01)providesasetofHPIdiscretecontrols,eacho

Preface7Shelves and modules supportedTheShelfManagerdescribedinthismanualisavailableontheseRadisysshelveswithasupportedfrontmodulein

4Changing Shelf Settings Using HPI70FRU power-on sequence commit status sensorTheshelfresourcecontainsanadditionalsensorcalledtheFRUpower‐on

4Changing Shelf Settings Using HPI71Querying the current hot-swap state of a FRUYoucanqueryaFRU’scurrenthot‐swapstate:•UsingtheCLIcommandst

4Changing Shelf Settings Using HPI72Overriding the default ShMS hot-swap mechanism WhetheraFRUsittinginaparticularslotisactivatedbytheShel

4Changing Shelf Settings Using HPI73Whileinmanualmode,youcanalsoupdatetheDelayBeforeNextPowerOnforthathardwareaddress.Thisisthean

4Changing Shelf Settings Using HPI74Overriding the cooling algorithmSystemmanagerscancompletelyoverridetheShelfManager’scoolingalgorithmand

4Changing Shelf Settings Using HPI75Overriding the cooling algorithm with an HPI applicationThefollowingstepsillustratehowyoucandothisusingH

4Changing Shelf Settings Using HPI76Changing shelf power propertiesTheshelfresourcecontainstwocontrolsthatenablesystemmanagerstoupdateshel

4Changing Shelf Settings Using HPI77HPI parameter controlTheshelfresources,alongwithallintelligentFRUresourcesandslotresources,supportthe

4Changing Shelf Settings Using HPI78HPI server parameter control configuration files Eachresourcehastwoconfigurationfiles:onefordefaultsettin

4Changing Shelf Settings Using HPI79How configuration files are created for a new resourceDuringHPIinitialization(whenresourcesarebeingcreated

Preface8Where to get more product informationVisittheRadisyswebsiteatwww.radisys.comforproductinformationandotherresources.Downloads(man

4Changing Shelf Settings Using HPI80Saving a configuration file to store custom settings for a resourceAfteraresourcehasbeenconfiguredtoadesir

4Changing Shelf Settings Using HPI81Error conditions TheparametercontroloperationsSAHPI_DEF A ULT_PARMorSAHPI_RESTORE_PARMwillfailandreturn

5Chapter82Managing Alarms and EventsThischapterdescribes:•TheShelfManager’sroleinmanagingalarms.•Howyoucanchangethealarmreportingbyass

5Managing Alarms and Events83Domain alarm types and troubleshooting overviewTable 8liststhealarmandsensortypesalongwithsampledescriptionsan

5Managing Alarms and Events84Threshold alarmsThefirsteightalarmtypesinTable 8arethresholdalarmswhichindicatethatathresholdsensordetect

5Managing Alarms and Events85UpperMajortemperatureeventsma yindicatethattheambienttemperatureistoohigh,inwhichca seyoumightneedtol

5Managing Alarms and Events86Resource utilization alarmsTheresourceutilizationalarmsindicatethattheShelfManagerisusingahighpercentageof

5Managing Alarms and Events87Domain alarm table entriesThealarmstringsarebasedontheassociatedsensorevents.Textwithinthealarmstringindic

5Managing Alarms and Events88Collecting information for the alarm configuration fileForeachtypeofFRUandeachsensorforwhichyouwill assignse

5Managing Alarms and Events89Table 11liststheinformationneededtoidentifythesensor,theeventstate,andthenewseveritytoassign.Table 11.

Preface9IPMIIntelligentPlatformManagementInterfaceSpecificationSecondGeneration,v2.0,DocumentRevision1.0,Intel,Hewlett‐Packard,NEC,Dell

5Managing Alarms and Events90Configuration file structure and contentsThe/etc/shmgralarm.conffileusesthisstructureforitscontents:FRUidentific

5Managing Alarms and Events91Alarm configuration procedureChangestothealarmconfigurationfiletakeeffectwhentheShelfManagerisrestarted.Thi

5Managing Alarms and Events92ATCA-6000 shelf display and alarm indicatorsTheshelfdisplaypanel(SDP)ontheATCA‐6000shelfconsistsofthefollowin

5Managing Alarms and Events93Figure 9. Format of alarm messages on the SDPSet to the location of the resource that has the alarm. The format is NNM fo

5Managing Alarms and Events94ATCA-6006 shelf alarm indicatorsTheoptionalshelfalarmpanelontheATCA‐6006shelfconsistsofthefollowing:•ShelfPo

5Managing Alarms and Events95ATCA-6014 and ATCA-6016 shelf alarm indicatorsTheATCA‐6014andtheATCA‐6016shelvesbothhaveashelfalarmpanel(SAP)

5Managing Alarms and Events96Thecontrolsforthetelcoalarmcontacts(AlarmCritical,AlarmMajor,andAlarmMinor)arestandardHPIcontrols.Onth

5Managing Alarms and Events97Parameter values for setting alarm controlsThefollowinglistidentifiesthepossiblesaHpiControlSet()functionparamete

6Chapter98TroubleshootingThischapterprovidesdetailedinformationonthefollowingissues:•AvailablediagnostictoolsforFRUandShelfManagerprob

6Troubleshooting99Using the Shelf Manager diagnostics utilityTheshmgrdiagutilitygathersinformationonthecurrentstateoftheShelfManager,the