Skip to content →

RPKI – Use Routinator with Cisco IOS-XR

While digging about how to drop invalid ROA, I tested Routinator setup. Installing Routinator RPKI-RTR Cache validator is pretty easy using their documentation.

curl https://sh.rustup.rs -sSf | sh
source ~/.cargo/env
cargo install routinator
routinator init
# Follow instructions provided
routinator server --rtr 127.0.0.1:3323

When this is done, you can then start configuration on the router. I almost work daily on Cisco IOS-XR platform (on ASR9K hardware). And in fact, there are some tricks to do for this to work, as IOS-XR support only RTR protocol over Secure Transport (SSH for example).

Configure RPKI server and secure transport

On the RPKI server, you should create a new user for SSH secure transport for RTR protocol

adduser rpki

Then you should setup a sub-system on sshd_config

# cat /etc/ssh/sshd_config
[...]
PermitRootLogin no
# needed for user RPKI
PasswordAuthentication yes
[...]
# Define an `rpki-rtr` subsystem which is actually `netcat` used to proxy STDIN/STDOUT to a running `routinator rtrd -a -l 127.0.0.1:3323`
Subsystem       rpki-rtr        /bin/nc 127.0.0.1 3323
[...]
# Certain routers may use old KEX algos and Ciphers which are no longer enabled by default.
# These examples are required in IOS-XR 5.3 but no longer enabled by default in OpenSSH 7.3
Ciphers +3des-cbc
KexAlgorithms +diffie-hellman-group1-sha1

When you’ve done this, we can move on the IOS-XR side to setup RPKI server.

Configure IOS-XR RPKI server

To configure IOS-XR, you’ll need first to setup RPKI server using SSH username and password (which will be not shown after commit in the configuration).

router bgp 64567
!
 rpki server 1.2.3.4
  username rpki
  password rpkipassword
  transport ssh port 22
  refresh-time 3600
  response-time 600

When this is done, you will need to setup SSH client, as yes, IOS-XR ssh client is still using Cisco SSH v1.99 protocol version ! You can also setup vrf source and interface source if needed. Take care, some releases, like eXR (IOS-XR x64 version) in 6.1.x will not support ssh client v2 option …

ssh client v2
ssh client vrf myVRF
ssh client source-interface Loopback10

Then after, connection should be established

bgp[1064]: %ROUTING-BGP-5-RPKI_ADJCHANGE : 1.2.3.4 UP

RP/0/RP0/CPU0:router#sh bgp rpki summary

RPKI cache-servers configured: 1
RPKI database
  Total IPv4 net/path: 97550/105601
  Total IPv6 net/path: 15818/17522

RP/0/RP0/CPU0:router#sh bgp rpki server 1.2.3.4

RPKI Cache-Server 1.2.3.4
  Transport: SSH port 22
  Connect state: ESTAB
  Conn attempts: 1
  Total byte RX: 4080600
  Total byte TX: 7652
SSH information
  Username: rpki
  Password: *****
  SSH PID: 674259340
RPKI-RTR protocol information
  Serial number: 727
  Cache nonce: 0x79CA
  Protocol state: DATA_END
  Refresh  time: 3600 seconds
  Response time: 600 seconds
  Purge time: 60 seconds
  Protocol exchange
    ROAs announced: 131296 IPv4   23152 IPv6
    ROAs withdrawn:  25695 IPv4    5630 IPv6
    Error Reports :      0 sent       0 rcvd

Then now, you can enable ROV on IOS-XR, based on the RPKI table

RP/0/RP0/CPU0:router#sh bgp rpki table

  Network               Maxlen          Origin-AS         Server
  1.0.0.0/24            24              13335             1.2.3.4
  1.1.1.0/24            24              13335             1.2.3.4
  1.6.132.240/29        29              9583              1.2.3.4
  1.9.0.0/16            24              4788              1.2.3.4
  1.9.12.0/24           24              65037             1.2.3.4
  1.9.21.0/24           24              24514             1.2.3.4
[...]

Enable Route Origin Validation on IOS-XR

As stated in the Cisco documentation : BGP Prefix Origin Validation Based on RPKI, and thanks to a Cisco SE, I’ve discover that “Starting from Release 6.5.1, origin-as validation is disabled by default, you must enable it per address family”.

router bgp 64567
 !
 address-family ipv4 unicast
  bgp origin-as validation enable
  bgp bestpath origin-as use validity
  bgp bestpath origin-as allow invalid
 !
 address-family ipv6 unicast
  bgp origin-as validation enable
  bgp bestpath origin-as use validity
  bgp bestpath origin-as allow invalid
 !

In fact, if you enable “bgp bestpath origin-as use validity“, you should take care on how the BGP Best Path Selection is modified. See Patel NANOG presentation about Cisco’s Origin Validation Implementation. Reading this, BGP will prefer Valid pathes over Not-known path (over Invalid ones if you allow it). It means eBGP paths received on iBGP sessions will probably will be removed sooner from Best Path Selection algorithm, even if Local-Pref or Med is preferred on iBGP received paths due to a higher priority on the tie break for RPKI ROV.

bgp bestpath origin-as use validity behavior

During BGP best path selection, the default behavior, if neither of the above options is configured, is that the system will prefer prefixes in the following order:
Those with a validation state of valid.
Those with a validation state of not found.
Those with a validation state of invalid (which, by default, will not be installed in the routing table).
These preferences override metric, local preference, and other choices made during the bestpath computation.

You should use the useful command to understand and check impact.

RP/0/RP0/CPU0:router# sh bgp 1.1.1.0/24 bestpath-compare

On my side, I prefer to drop invalid using route policies on the eBGP sessions, so I can keep control. So I do not use bestpath validation :

router bgp 64567 bgp origin-as validation time 30
router bgp 64567 address-family ipv4 unicast bgp origin-as validation enable
router bgp 64567 address-family ipv4 unicast bgp bestpath origin-as allow invalid
router bgp 64567 address-family ipv6 unicast bgp origin-as validation enable
router bgp 64567 address-family ipv6 unicast bgp bestpath origin-as allow invalid

To drop invalid on each eBGP sessions, I simply use the following standard route-policy :

route-policy RP_DROP_RPKI_INVALID
  if validation-state is invalid then
    drop
  endif
end-policy

This RPL is called at start when dropping some Bogons Prefixes (aka Martians) or ASN.

route-policy RP_EBGP_PEER_IN
  apply RP_DROP_BOGONS
  apply RP_DROP_DEFAULT_ROUTE
  apply RP_DROP_RPKI_INVALID
  [...]
end-policy

Then you’ve done 😉 Next article : how to setup Routinator with configuration file and SLURM exceptions file.

Published in Réseau

11 Comments

  1. Tiago Setti Tiago Setti

    Nice post. I think it may have a typo on this line:
    if you enable “bgp origin-as validation enable“, you should take care on how the BGP Best Path Selection is modified….

    Actually, the “bgp origin-as validation enable“ does not change any BGP Best Path behavior, the command that does that is this one “bgp bestpath origin-as use validity”

    So by default, the IOS XR does not change any bestpath behavior, even with BGP origin validation enabled. BUT, if you wish to do so, you can include the command “bgp bestpath origin-as use validity” and then IOS XR will not use INVALID prefixes in the bestpath selection.

    • Fabien Vincent Fabien Vincent

      You’re right ! Thanks for the remark, will modify it in few minutes.

  2. Alex Alex

    Hello Fabien,
    Could you tell or share you experience, how does influence turn on RPKI functionality affect on routers CPU?
    I turned on RPKI drop for Invalid via apply policy per BGP neighbor on two ASR9K border routers and see strange picture.
    You can see my results here: http://golas.ru/rpki-validation-2.png

    • Fabien Vincent Fabien Vincent

      Well I didn’t see anything like this, but perhaps I could ask my old colleague to check if it’s the same. 2 questions to understand :

      • What’s the polling time ? Dela between 2 CPU measurements ?
      • What’s the A9K ? I did it on RP2 and did get anything like this, but each A9K / RSP or RP have different processors
      • Alex Alex

        Polling time is 5 minutes. Represented graph is CPU utilisation for the last day from NMS Cacti.
        The routers are Cisco ASR9001.

        I use two different RPKI local cache projects – FORT and Routinator.
        At first I’m thinked that one of the project influence on CPU. I turned off one of the projects one by one. Isn’t dependent with which project to work, the situation almost the same.
        Now I started change timers – “refresh-time” and “response-time” on one of the border gateway. It was 300 and 120 seconds accordingly, and now timers according recommendations RFC8210 3600 and 600 seconds. Situation almost isn’t change.
        After changed timers: http://www.golas.ru/rpki-validation-timers.png

        • Fabien Vincent Fabien Vincent

          ASR9001 has small PPC CPU … It can explain perhaps by BGP Scanner processes ?

          • Alex Alex

            I doubt. BGP generic scan interval and per AF scan interval is 60 seconds by default.
            I increased timers in RPKI local cache project and will be monitor that will happen.

          • Alex Alex

            Share information for others.
            After increase refresh timer to 4h on both RPKI local cache projects, I received following result:
            http://www.golas.ru/rpki1-mem-cpu-2.png
            On this picture you can see correlation between memory usage by Routinator and CPU peaks.
            It seems you is right according CPU performance ASR 9001, I will try to consult with vendor.

          • Fabien Vincent Fabien Vincent

            Glad to push you in the right direction. ASR9001 has small PPC CPU, so it seems logic CPU usage is higher.

Comments are closed.

fr_FRFR