Ever heard of streaming telemetry?
No?
Would you like to?

We no longer have to poll network devices for thousands of data points every few seconds to monitor them, we can just have them report back to us whatever data we want, either on a timer, or when it changes.

Configuration is disarmingly easy.

On your Cisco IOS-XE device you will need to add the following line to enable YANG-based configuration:

netconf-yang

And then to get a report of incoming and outgoing traffic, every 60 seconds, for all interfaces, just add:

telemetry ietf subscription 2
 encoding encode-kvgpb
 filter xpath /interfaces/interface/statistics/rx-kbps
 stream yang-push
 update-policy periodic 6000
 receiver ip address <telegraf_server_ip> 57000 protocol grpc-tcp
telemetry ietf subscription 3
 encoding encode-kvgpb
 filter xpath /interfaces/interface/statistics/tx-kbps
 stream yang-push
 update-policy periodic 6000
 receiver ip address <telegraf_server_ip> 57000 protocol grpc-tcp

Then on your Influxdb web page create a new Telegraf config for “Cisco Model-driven telemetry”, copy the two lines you’re prompted for in a terminal – and you’re done!

No, really. That’s it.
And you know the best part? This is an open standard, so you will be able to use this across all your machines that support YANG-based streaming telemetry.

Where to get monitoring paths

While “/interfaces/interface/statistics/rx-kbps” is definitely less cryptic than SNMP’s 1.3.180.12.67.30… equivalent, one might wonder where you can get those paths from.

Enter Cisco YANG suite.

Install it locally (you need Docker, it won’t work on Podman, I tried) following the guides

Create a new device, create a new repo, and either import data directly from your own machine or from the public Git repo for YANG, like so:

For example, let’s say we want to monitor power supply status.
Go under Explore -> Yang.
We’re looking for something called power so… power?

Yes!

..no.
Alright, this is not an exact science. What about going letter by letter… “po”?

Hey look!

Let’s say we want to get all the juicy data from “poe-switch”.
Well it’s just a matter of getting its xpath…

And putting it in the conf

telemetry ietf subscription 5
 encoding encode-kvgpb
 filter xpath /poe-oper-data/poe-switch
 stream yang-push
 update-policy periodic 100
 receiver ip address 10.164.85.152 57777 protocol grpc-tcp

And after a little while…

This looks _very_ promising. Maybe we could also set it to send data “on-change” instead of every 10 seconds, although even with all this, CPU usage is still below 1%

Sure, but did you know my Python library netwalk has support for configuring spanning-tree on Cisco switchports and Ansible does not?

Someone should do something about it.
Anyone?
Fine, I’ll do it myself.

In this post, I truly have absolutely no idea what I’m doing.
And I can’t wait to begin.

Ansible galaxy already has an official-looking package for cisco network devices, so let’s clone it and open it in an IDE.

I’ll try to follow the parsing of “l2 interfaces”, which manages assigning vlans and mode access/trunk

Ok, that’s a lot of files with the same name.

What do they do?

Let’s open them and figure out. Luckily they are all well-commented.

• argspec is an autogenerated file containing the data structure you can pass to the library
• config, as per its description, is the file “where the current configuration (as dict) is compared to the provided configuration (as dict) and the command set necessary to bring the current configuration to its desired end-state is created.
• facts: It is in this file the configuration is collected from the device for a given resource, parsed, and the facts tree is populated based on the configuration.
• rm_templates: a list of parser definitions and associated functions that facilitates both facts gathering and native command generation for the given network resource.
• modules is the main entry point for module execution

So by logic, when we connect to a device we call something inside facts, which uses parsers defined in rm_templates to generate a dict that is used by config to find the differences and generate the appropriate commands to bring the config up to spec.

Unfortunately I haven’t been able to have VS Code debug this collection correctly (it hangs at gathering facts) so let’s

We’re going to start by writing a single test, copying test_l2_interface and crudely swapping all mentions of “l2” with “stp”, so the only thing we have to worry about is

    def test_ios_stp_interfaces_merged(self):
        self.execute_show_command.return_value = dedent(
            """\
            interface GigabitEthernet0/1
            """,
        )
        set_module_args(
            dict(
                config=[
                    dict(
                        bpduguard=True,
                        name="GigabitEthernet0/1",
                    ),
                ],
                state="merged",
            ),
        )
        commands = [
            "interface GigabitEthernet0/1",
            "spanning-tree bpduguard enable",
        ]
        result = self.execute_module(changed=True)
        self.maxDiff = None
        self.assertEqual(result["commands"], commands)

Of course our linter has flared up like crazy because of all the missing dependencies, so let’s fix that too.
For now, I’ll limit to parsing whether bpduguard is True or False.
Is this compliant to Ansible best practices? I have no idea, I’m just throwing code at the interpreter to see what sticks.

Let’s create those files, always copying from their “l2” correspective.

OK I think I’ve found something interesting.
module_utils/network/ios/rm_templates/stp_interfaces.py contains a few lines that look like what I was looking for

    PARSERS = [
        {
            "name": "name",
            "getval": re.compile(
                r"""^interface
                    (\s(?P<name>\S+))
                    $""",
                re.VERBOSE,
            ),
            "compval": "name",
            "setval": "interface {{ name }}",
            "result": {"{{ name }}": {"name": "{{ name }}"}},
            "shared": True,
        },
        {
            "name": "access.vlan",
            "getval": re.compile(
                r"""
                \s+switchport\saccess\svlan\s(?P<vlan>\d+)
                $""", re.VERBOSE,
            ),
            "setval": "switchport access vlan {{ access.vlan }}",
            "remval": "switchport access vlan",
            "result": {
                "{{ name }}": {
                    "access": {
                        "vlan": "{{ vlan }}",
                    },
                },
            },
        },

Looks like a set of regexes to parse the configuration, where each defines the path of the data to generate, a regex describing how to get the data, the command to set the desired config (setval), the command to remove the config (remval) and the result to return once the module is done.

Let’s write the first parser

{
    "name": "bpduguard",
    "getval": re.compile(
        r"""
        \s+spanning-tree\sbpduguard\s(?P<bpduguard_status>\w+)
        $""", re.VERBOSE,
    ),
    "setval": "{{ 'spanning-tree bpduguard enable' if bpduguard == True}}",
    "remval": "no spanning-tree bpduguard",
    "result": {
        "{{ name }}": {"bpduguard": "{{ 'True' if bpduguard_status == 'enable' else 'False' }}"},
    },
},

Moment of truth… let’s run a simple playbook set to “gather” the configuration, so it will parse it for us

Well! Looks like I only need to… parse… the entire… command tree… for spanning tree.
Remind me again why I even began doing this?

Anyway, the result of this post is available in this commit