Skip to content

WIP conversation starter for OPTE + Sled Agent#377

Closed
rzezeski wants to merge 9 commits into
mainfrom
rpz-opte-sled-agent
Closed

WIP conversation starter for OPTE + Sled Agent#377
rzezeski wants to merge 9 commits into
mainfrom
rpz-opte-sled-agent

Conversation

@rzezeski

@rzezeski rzezeski commented Nov 10, 2021

Copy link
Copy Markdown
Contributor

This is a conversation starter for work towards #247.

I have not tested this nor even tried to compile it. I just wrote some
code on the fly to serve as a straw man for how I think Sled Agent +
OPTE interaction might look for the next demo milestone. In this
commit I only cover the registration/unregistration of the OPTE Port,
which is what overlays the illumos VNIC. Once again, this is messy and
stream of conciousness. Just wanna get the juices flowing in terms of
integration.

@smklein Hopefully this sheds some light on what I had in mind. Also, hopefully you have an idea on where we might want to put calls to add/remove firewall rules.

TODOs

  • Modify OPTE to make SNAT optional (and thus remove the need for much of this information for now)
  • Figure out where to put firewall interactions.
  • Figure out how to expose opteadm outside of the opte repo, particularly how to have sled agent consume it.
  • Are VLANs on the scene in the lab demo env? If so need to add that support to OPTE (mostly make sure it does parse it as a normal ethernet header)
  • Figure out how to run sled agent + propolis + OPTE on a host and test this at home.

I have not tested this nor even tried to compile it. I just wrote some
code on the fly to serve as a straw man for how I think Sled Agent +
OPTE interaction might look for the next demo milestone. In this
commit I only cover the registration/unregistration of the OPTE Port,
which is what overlays the illumos VNIC. Once again, this is messy and
stream of conciousness. Just wanna get the juices flowing in terms of
integration.
@rzezeski rzezeski added the networking Related to the networking. label Nov 10, 2021
@rzezeski rzezeski self-assigned this Nov 10, 2021
@smklein

smklein commented Nov 10, 2021

Copy link
Copy Markdown
Collaborator

This is a conversation starter for work towards #247.

I have not tested this nor even tried to compile it. I just wrote some code on the fly to serve as a straw man for how I think Sled Agent + OPTE interaction might look for the next demo milestone. In this commit I only cover the registration/unregistration of the OPTE Port, which is what overlays the illumos VNIC. Once again, this is messy and stream of conciousness. Just wanna get the juices flowing in terms of integration.

Looks like a good start. FYI, "cargo build" should basically work out of the box on Helios for omicron, so if that isn't working, lmk and I'm happy to help debug.

@smklein Hopefully this sheds some light on what I had in mind. Also, hopefully you have an idea on where we might want to put calls to add/remove firewall rules.

The sled agent uses a structure called InstanceManager, which acts basically like a map of "instance UUID -> instance object". Currently, adding an instance through the HTTP interface basically adds something to this map, spinning up the instance along the way.

I figured that:

  1. On instance creation, we can expand the arguments passed to the Sled Agent to include relevant firewall rules,
  2. We can add a new HTTP endpoint to allow modification of firewall rules later too.

TODOs

  • Modify OPTE to make SNAT optional (and thus remove the need for much of this information for now)
  • Figure out where to put firewall interactions.
  • Figure out how to expose opteadm outside of the opte repo, particularly how to have sled agent consume it.

Would a direct dependency on the library not work? That's how we're depending on e.g. Propolis.

  • Are VLANs on the scene in the lab demo env? If so need to add that support to OPTE (mostly make sure it does parse it as a normal ethernet header)
  • Figure out how to run sled agent + propolis + OPTE on a host and test this at home.

Comment thread sled-agent/src/instance.rs Outdated
let name = guest_vnic_name(allocator.next());
Dladm::create_vnic(physical_dl, &name, mac, vlan)?;

// TODO (rpz): If vlan is Some(N), then OPTE is going to have

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This VLAN tagging was added because of this comment: #186 (comment)

@jclulow can confirm, but I believe VLANs are in use in the lab environment.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I totally forgot that mac will strip off the VLAN before I see the packet. So OPTE doesn't have to worry about this at all.

Comment thread sled-agent/src/instance.rs Outdated
_ => return Err("OPTE only supports IPv4 guest IPs at the moment"),
};

let public_mac = match mac {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's going on here? Why modify half the MAC address?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was just a convenient way to get a new mac address that was easily related back to the private mac (those three bytes represent the Oxide OUI). This is going away for the demo as I'm making SNAT optional (and this really was just a hack to work around my home router).

@bnaecker

Copy link
Copy Markdown
Collaborator

I'm going to close this, since the work was actually implemented and merged in #955.

@bnaecker bnaecker closed this May 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

networking Related to the networking.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants