opaxlattopology (8) Linux Manual Page

opaxlattopology Generates a topology XML file of a cluster using topology.csv, linksum_swd06.csv, and linksum_swd24.csv as input. The topology file can be used to bring up and verify the cluster. NOTE: The topology.csv input file must be present in the same directory from which the script operates, but the linksum CSV files are read from the /usr/lib/opa-ff/samples directory.

Syntax

opaxlattopology [-d level-v level-i level-c char-K -?]
[source [dest]]

Options

-d level

Output detail level. Default = 0. Levels are additive.

By default, the top level is always produced. Switch, rack, and rack group topology files can be added to the output by choosing the appropriate level. If the output at the group or rack level is specified, then group or rack names must be provided in the spreadsheet. Detailed output can be specified in any combination. A directory for each topology XML file is created hierarchically, with group directories (if specified) at the highest level, followed by rack and switch directories (if specified).

1 Intel(R) Omni-Path Edge Switch 100 Series topology files.

2 Rack topology files.

4 Rack group topology files.

-v level

Verbose level. Range = 0 – 8. Default = 2.

0 No output.

1 Progress output.

2 Reserved.

4 Time stamps.

8 Reserved.

-i level

Output indent level. Range = 0 – 15. Default = 0.

-c char

NodeDesc concatenation character. Used when creating NodeDesc values (that is, Name to Name-2, Name to HFI-1, and so on). A space is used by default, but another character, such as underscore, can be specified.

-K

Does not clean temporary files. Prevents temporary files in each topology directory from being removed. Temporary files contain CSV formatted lists of links, HFIs, and switches used to create a topology XML file. Temporary files are not typically needed after a topology file is created, however they are used for creating linksum_swd06.csv and linksum_swd24.csv files, or can be retained for subsequent inspection or processing.

-?

Prints this output.

Description

The opaxlattopology script reads the topology.csv file from the local directory, and reads the other files from /usr/lib/opa-ff/samples/linksum_swd06.csv and /usr/lib/opa-ff/samples/linksum_swd24.csv. The topology.csv file is created from the topology.xlsx spreadsheet by saving the Fabric Links tab as a .CSV file to topology.csv. A sample topology.xlsx is located in the /usr/lib/opa-ff/samples/ directory. Inspect the topology.csv file to ensure that each row contains the correct and same number of comma separators. Any extraneous entries in the spreadsheet can cause the CSV output to have extra fields.

The script outputs one or more topology files starting with topology.0:0.xml. The topology.csv input file must be present in the same directory from which the script operates, but the linksum CSV files are read from the /usr/lib/opa-ff/samples directory.

Example

opaxlattopology
# reads default input ‘topology.csv’ and creates default
# output ‘topology.0:0.xml’

opaxlattopology fabric_2.csv
# reads input ‘fabric_2.csv’ and creates default output

See topology.xlsx for examples of links between HFI and Edge SW (rows 4-7), HFI and Core SW (rows 8-11), and Edge SW and Core SW (rows 12-15).

Environment Variables

The following environment variables allow user-specified MTU.

MTU_SW_SW

If set, it overrides default MTU on switch-to-switch links. Default = 10240

MTU_SW_HFI

If set, it overrides default MTU on switch-to-HFI links. Default = 8192

Creating linksum Files

The linksum_swd06.csv and linksum_swd24.csv files are provided as stand-alone files in the /usr/lib/opa-ff/samples directory. However, they can be recreated (or modified) from the spreadsheet, if needed, by performing the following steps:

1)

Save each of the following from the topology.xlsx file as individual .csv files:

•

Internal SWD06 Links tab as linksum_swd06.csv

•

Internal SWD24 Links tab as linksum_swd24.csv

•

Fabric Links tab as topology.csv

2)

For each saved topology.csv file, run the script with the -K option.

3)

Upon completion of the script, save the top level linksum.csv file as linksum_swd06.csv or linksum_swd24.csv as appropriate.

Including SM Blocks in Topology

You must manually edit the /etc/sysconfig/opa/topology.0:0.xml file to include SM blocks for each SM in your fabric. opaxlattopology translates the topology.csv file which does not include SM blocks in the xml file by default. If manual edits are not completed, then the command opareport -o verifyall -T topology.0:0.xml fails with errors similar to the following:# opareport -o verifyall -T topology.0:0.xml
SMs Topology Verification

SMs Found with incorrect configuration:
NodeGUID Port PortGUID Type Name
0x0011750101671ed9 1 0x0011750101671ed9 FI phkp3un86 hfi1_0
NodeDetails: koplab
Unexpected SM

1 of 1 Fabric SMs Checked

SMs Expected but Missing or Duplicate in input:
0 of 0 Input SMs Checked

Total of 1 Incorrect SMs found
0 Missing, 1 Unexpected, 0 Duplicate, 0 Different

To retrieve the info for the SM block, use the command opareport -o nodes -F sm as shown in the following example.
# opareport -o nodes -F sm
Getting All Node Records…
Done Getting All Node Records
Done Getting All Link Records
Done Getting All Cable Info Records
Done Getting All SM Info Records
Node Type Summary
Focused on:
Port: 1 0x0011750101671ed9
in Node: 0x0011750101671ed9 FI phkp3un86 hfi1_0

3 Connected FIs in Fabric:
Name: phkp3un86 hfi1_0
NodeGUID: 0x0011750101671ed9 Type: FI
Ports: 1 PartitionCap: 16 SystemImageGuid: 0x0011750101671ed9
BaseVer: 128 SmaVer: 128 VendorID: 0x1175 DeviceID: 0x24f0 Rev: 0x10
1 Connected Ports:
PortNum: 1 LID: 0x0001 GUID: 0x0011750101671ed9
Neighbor: Name: phedfim20
NodeGUID: 0x0011750102648205 Type: SW PortNum: 46
Width: 4 Speed: 25Gb Downgraded? No
1 Matching FIs Found

1 Connected Switches in Fabric:
0 Matching Switches Found

1 Connected SMs in Fabric:
State: Master Name: phkp3un86 hfi1_0
NodeGUID: 0x0011750101671ed9 Type: FI
PortNum: 1 LID: 0x0001 PortGUID: 0x0011750101671ed9
SM_Key: 0x0000000000000000 Priority: 0 ActCount: 0x0000bcb2
1 Matching SMs Found