Restricted account type proposal

Ideas RFCs
,
1

Originally posted at SPHERE design google drive by @mirkovic, reposted here by @jdbarnes for discussion/comment

Overall philosophy

We may regard SPHERE users as falling into two categories:

  1. Full: Those that use SPHERE for research and are leveraging full functionalities
  2. Restricted: Those that use SPHERE as part of another activity (class, demo, evaluation of artifacts) as a tool, and need limited functionalities

Differences between full and restricted accounts

Full accounts are expected to use the full suite of our user interfaces, GUI, SSH and Jupyter and should be able to have full control over their account. They should be able to reset passwords, update accounts, and close an account if they do not want to use us anymore. They should be able to select their username if it is not currently taken. They are expected to have one account per email address.

Restricted accounts are expected to use some other UI (class, EAC, demo) and interact with SPHERE in limited ways. They are created programmatically by org owner, they live for a limited time and are then recycled to be used by new users. They get a programmatically generated username and password and the org owner has access to everyone’s passwords. Users cannot change their password, but the org owner can give them a new programmatically generated one if needed. It is useful for users to get their username and password via email. Users may have multiple accounts with a single email address, if they belong to multiple orgs at the same time.

Activity Full account Restricted account
Creation Via Web UI or mrg CLI Bulk by org owner
Username Chosen Programmatic
Password Chosen Programmatic
Multiple accounts per email No Possible
Password reset Via email By org owner via special UI
Reject resets via our form
Password change notification No By email
Login to SPHERE Normal via our UIs May be through special UI
Lifetime As needed Until org owner recycles
Account open checks Reject on existing email Create email alias on existing email

SUPPORT NEEDED: Highlighted requirements necessitate us to support email aliases. On a restricted account open we’d check if an email is already in use. If so, we’d create an alias and store it in a table that remembers username, original email and alias email. Alias is passed to Kratos. Password change on a restricted account gets emailed to user (we can do this from class or AEC UI, no need for special support on Merge). We set up alias accounts to forward to the original email address.

Thought experiment: what if all accounts had full access?

Activity Now Thought experiment
Creation Bulk by org owner By user via Web UI
Username Programmatic (usc430aa, usc430ab, usc430ac) Chosen (sunshine, jross, mattw)
Password Programmatic Chosen
Password reset By org owner via special UI Via email and our reset form
Login to SPHERE Normal via our UIs May be through special UI
Lifetime As needed Until org owner recycles, but special checks needed to see if user belongs to another org or project before recycling
Account open checks Reject on existing email Reject on existing email
Pre-create accounts Yes No
Chance of idle accounts Low Very high
Chance of support requests Low Very high
Statistics Account creation logs Org join logs
Use mode statistics (research/edu) Easy Very hard

This is possible but messy. Lots of actions that were automated are now user-driven, which opens space for user mistakes and support tickets that require prompt handling due to class, AEC, demo deadlines. Lots of user requests for support will come to us (reset email cannot be found, etc.) Org owner now cannot log in as a given user to debug issues, because they have no access to user password (and neither can we). We would also need special support by ops to create accurate statistics on class vs research users and use times.

2

Chris mentioned that we could simply use plus in email addresses like sunshine+user2@isi.edu and that this should work for most mail services (it just ignores string after +) and we should test if it works for Kratos. From Jelena’s writeup we do not really need special table then.

Geoff mentioned that Kratos supports restricted accounts that cannot require password recovery via email and that would work perfectly for our restricted accounts. So minimal to no changes would be needed.

3

== Brian’s points and possible workarounds
Thank you very much for writing this up. I appreciate you going through the thought experiment, even if your inclination is against doing things that way.

I want to ask about some of the issues you identified in the thought experiment

Chance of idle accounts

Since restricted accounts are associated with an organization, we could develop the necessary org controls to ensure that no restricted accounts or associated data (experiments, projects, XDCs, etc.) are leaked on organization cleanup. A good number of these methods already exist in the code.

In the common case where a restricted account is associated with only one organization, this would be very straightforward – i.e., remove all account projects, experiments, XDCs, etc (or whatever the removal policy is).

In uncommon cases where accounts are associated with multiple organizations, we would have to consider what the appropriate policy is here. e.g., you may want to only remove experiments associated with the organization, not all of the experiments the user has access to. Whatever the policy, I think it’s doable, mainly because essentially all portal data is already namespaced by project, and projects themselves can be tied to organizations. Thus it would be straightforward to only clean up the data associated with projects that belonged to the organization, and let the rest of the user’s projects, which belong to other organizations, can remain.

There may be some tricky edge cases here to consider, but overall it doesn’t feel like a major task. The notion of organizations in the portal was designed for this type of thing.

Chance of support requests

I think the critical piece here is to make sure that organization owners have all of the permissions and visibility needed to handle as many issues as possible.

In general, any issue related to visibility or control of user data can be easily accommodated by the role-based access control mechanisms that currently exist in the portal. e.g., this line here denotes that org maintainers can always read account data (even data marked ‘private’) in their constituent users’ accounts. We could configure policy so that org owners can always access anything that their constituents have access to.

The one piece that org owners probably could not do is reset account passwords, given the kratos reliance on email to do so. However, it may be possible to let org owners perform the resets by sending reset links to both the account email and the email of the org maintainer – we would need @glawler to investigate)

I see why the current programmatic approach is nice here; however, if our password reset mechanism is robust and easily visible, I would hope that all users could navigate it without issue. We could do the standard thing which is put a link to reset password on the login page, perhaps only making it visible after a failed login attempt.

Use mode statistics

This one seems straightforward to me, which again stems from the fact nearly all portal data is namespaced by project, and that each project can be namespaced within an organization. Either the project and/or the organization would be categorized appropriately (research, edu, AEC, etc.) thus giving us the insight we need.

4

Lastly, I wrote up a section in the Merge docs which diagrams how Merge objects relate to each other. At one point, this had a section on managing classes, but it was removed to avoid confusing users given that we were sticking with the DeterLab system for the time being

5

== Jelena’s concerns about workflow for classes and demos

You’re right that all this is possible to do. It is perfectly possible, as you outlined to work around various issues.

I just dislike this solution for the following reasons: (1) it is messy - when seeing an account name ‘ericf’ we no longer know if this is research or class user, (2) it requires more work on our end to implement - the way it is now I can transfer everything to new platform easily, just need some help with GUI. With your suggested solution I will need more ops time to also add all the controls in the backend. (3) it puts more work on students, which means more questions for us. (4) we and instructors can no longer debug student issues because we cannot login as a student. (5) instructors are already used to what we do so far, so they have to change their workflows too. (6) it complicates demos - I can no longer pre-create accounts with set passwords for 30 people and just hand them over in a demo.

On the other hand, the solution we have works now, it is portable to our new platform and really does not require more than one small change to check if there’s a duplicate mail and introduce the alias into our system. From that point on demos work, HotCRP works, classes work and we only have GUI work to port education functionalities to our new platform.

Does this help? Or if you still feel we should have one account per user and use it in mixed manner, perhaps this is the first topic to discuss in our design meeting? We would just have to schedule it ASAP since we will need to work on class functionality migration over summer and it also would impact HotCRP accounts.

6

(1) it is messy - when seeing an account name ‘ericf’ we no longer know if this is research or class user,

Well to me, the current approach is actually messier, even if it may be convenient given the current interfaces. My preferred approach is always to explicitly structure the data type on the backend to track the relevant information (such as account type), and then make it easily queryable through the user interfaces – i.e., mrg list users --type edu (or some such), or a dropdown list on the GUIs.

The current approach encodes semantics of the account type into the naming scheme. This is fine but it introduces a constraint that Merge is not aware of anywhere – anyone could happily come along and create an account that matches this regex and then run the risk of polluting statistics, not to mention possibly getting caught in automated recycle processes.

I don’t think we need to belabor this issue, it’s just a difference in our tastes. Perhaps we do both things: 1) enforce restrictions on how restricted accounts are named, and 2) explicitly track account types.

(2) it requires more work on our end to implement - the way it is now I can transfer everything to new platform easily, just need some help with GUI. With your suggested solution I will need more ops time to also add all the controls in the backend.

It may require more work, but I am not so sure about that. As I said, most of what I propose above already exists in Merge. What you propose seems simple enough but that will be a question for Geoff to answer once he’s dived into kratos internals. I also don’t know what TBD constraints will arise on how we have to handle IDs, authorization, account resets etc. once we implement CILogon support.

(3) it puts more work on students, which means more questions for us.

We are just talking about clicking a button to create an account, correct? Am I missing something?

Admittedly, I do tend to overlook how things I think are simple are not always simple for students.

(4) we and instructors can no longer debug student issues because we cannot login as a student.

We cannot login as students, but as I said, we can login as ourselves and, via the Merge policy layer, be given permission to see exactly what they see and debug issues from their perspective.

Think about this as akin to ssh briankoco@system, followed by sudo su - mirkovic. I didn’t login as you, but I can see all of your data. Just about the only thing I could not directly debug are the actual login or SSH issues you might have been experiencing.

As I noted, this already mostly exists, I am not proposing to build any complex new functionality.

(5) instructors are already used to what we do so far, so they have to change their workflows too.

(6) it complicates demos - I can no longer pre-create accounts with set passwords for 30 people and just hand them over in a demo.

Point taken on both of these.

On the other hand, the solution we have works now, it is portable to our new platform and really does not require more than one small change to check if there’s a duplicate mail and introduce the alias into our system. From that point on demos work, HotCRP works, classes work and we only have GUI work to port education functionalities to our new platform.

I understand this, but getting something to “work” on the system is only part of the task. We also need to make sure we can operate it and maintain it, and possibly extend it as needed in the future. While the points I’m raising above might seem like me making a mountain out of a mole-hill, it’s all oriented towards trying to abide by the way in which Merge is designed, so that people like Joe can maintain it.

I agree that auto-generated restricted accounts have some very nice uses, especially around bootstrapping demos. I’ve seen demos where ~30 mins of a ~60 min demo slot is spent getting accounts created for people. And I can see some of the benefits for classes. So I am not really pushing back against that at all.

7

Design questions from Brian

About restricted users:

  • must they always be assigned to an organization? Jelena: Yes
  • should they be able to join more than one organization? Jelena: No
  • if they are in an organization, should they always be deleted when the organization is deleted? Jelena: Yes. They are actually recycled and used for future members of that same org.
  • are there any other restrictions on what they can do? can they create projects? etc Jelena: I think the only restrictions are around login and email reset. Everything else should be as normal users.

About organizations:

  • can an organization include both restricted and un-restricted accounts? Jelena: it has org owner as unrestricted. Other should be restricted. Although I see no problem if there’s another or a handful unrestricted users that want to join e.g., co-teachers, TAs (although they also get programmatic accounts), co-presenters for demo. These accounts are not recycled, bc. the system keeps track of programmatically created accounts and only recycles those.
  • what happens on org deletion: do only restricted accounts get deleted while full accounts survive? Jelena: so far no org deletions. There are recycle events at the end of a class or demo or conference. At that point restricted accounts are deleted and full accounts are not touched.
  • if someone owns both a full account and a restricted account, is there a need to allow them to move data (e.g., experiments or just raw data on the mergefs) between them? Jelena: yes it would be nice to have this. But if it is complicated they can always move it to their platform as one user and then move into Merge as another user. I’m aware this is not ideal for large data, just offering it as a workaround if we need it.
8

Recycle account == delete and recreate with the same ID and new password?

What does it mean to recycle an org?

9

We don’t recycle orgs. We just recycle all accounts when a given class offering is up or a demo is done on a given project. We could do the same for conferences in which case they would end up having an organization like “acsac” versus “acsac2024” and their PI would change from year to year as a new person takes on the artifact evaluation.

10

Ok, makes sense.

One thing that I’m not seeing with restricted accounts: how are password recoveries/resets going to work, given that there is no longer a guaranteed email association with the account?

As far as I know, email based recovery is the only solution kratos offers for resetting a password. It would not be a good situation for neither org owner nor restricted user to be able to recover a forgotten password (this is the case currently), and I don’t think relying on a list of credentials generated randomly and never stored anywhere that Merge knows about is a good solution.

I’d also think that a restricted account owner should be able to change their password should they desire, even if they haven’t forgotten it. This wouldn’t affect any of our ability to see or modify their user state and aid in debugging, as described above, but would improve usability of the system for users

I suppose restricted account creds could be stored in an internal Merge database, completely outside of kratos, so that maintainers could look them up as needed. This would mean that maintainers can always recover the password, but does not provide any way to change it since that would still need to go through kratos somehow.

I believe we currently handle resets by completely wiping out the kratos ID for the account and then re-registering. The following flow would then work for resetting a pw:

  1. Login with current password (org maintainer can lookup via Merge as needed)
  2. Reset the password through a new API call which supports only restricted accounts. This call internally removes and then re-registers the ID to kratos with the new password. This call would be available to both the org maintainer as well as authenticated restricted account owner.
  3. After re-registering to kratos, this call would update the credentials table stored in the organization internal database.

Perhaps this is an acceptable solution after all?

Any thoughts on all of this @jdbarnes @glawler ?

11

Kratos v1, which we are now using, has an admin API to reset the password I believe. So I may be able to write a utility and install it somewhere in the portal to allow admins to arbitrarily set passwords for identities.

12

One small point. The idea of sending passwords in the clear in emails makes my skin crawl. We are cyber security researchers - we should be able to come up with a way that is secure or at least not laughably insecure. Even if these are limited accounts, no reason to do the wrong thing here.

13

ah, excellent

14

I agree - besides the security faux pas is the fact that we may be better off making a clean split from email integration on these restricted accounts. @sunshine mentioned there was some sort of restricted identity type in kratos that may make sense for this - do you have a pointer to that?

15

This is the documentation: What is an identity in Ory? | Ory. I cannot find good information and have not had great luck in using multiple identity schemas in a single kratos instance though.

16

ack. A cursory search of the kratos issues shows that multiple id schemas is a desired feature but not yet implemented:

One alternative would be to setup a 2nd kratos instance with a different schema for restricted accounts

17

Yeah, multiple kratoses might work. I suppose that would then mean multiple databases and updates to launch (or a new GUI) to use the different schemas.

18

I have a proposal to at least consider as a thought exercise.

Start by considering that the portal includes a single unrestricted blanket organization which will be the home of all unrestricted accounts. This could be something like testbed-users. Users that join the testbed through the the regular account creation process would be admitted to this organization. Only Merge portal ops would be maintainers of this org.

What if rather than associating a subset of accounts as restricted, we associate a subset of organizations as restricted?

Consider a restricted organization org123. It would have the following properties:

  1. Accounts can be admitted to org123 through the existing process - selection via a dropdown list on the launch registration page, or via a flag on the command line.
  2. Organizations can, optionally, have windows of “open enrollment”. During this time, any account requesting membership in the organization will be granted.
  • Enrollments could also be accompanied by a single shared registration code that can be provided at registration time to authorize registration. This could be shared with students or demo users in person during bootstrapping.
  • The primary use-case here is around bootstrapping demos – getting a new user onto a demo becomes nothing other than registering on launch and selecting org123 , and passing the code with the registration
  1. A user that is admitted to a restricted organization can request membership both in (1) other restricted organizations (e.g., if they are enrolled in multiple classes or demos), and/or (2) in the testbed-users organization.
  2. On deletion/archival of org123, a check happens. All users in the organization are assessed for membership in other organizations. If they are not members or any other organization, their accounts get deleted, and other resources they created are cleaned up (e.g., projects, experiments, XDCs). If they are members of another organization, their accounts survive overall on the testbed.

The benefits of this approach:

  • Most of this logic is implemented in the portal already. The minor additions needed are for automated account removal on organization teardown/recycle, and addition of the open enrollment logic
  • This requires no special handling of a new class of accounts. No need to change how we use kratos, or setup a separate kratos instance, or maintain a table of email aliases with forwarding logic.
  • We are able to cleanup the accounts that are only associated with short-duration events such as demos or classes.
  • Users that are members in multiple restricted organizations only need a single account with a single email address. e.g., students that are enrolled in multiple courses need only a single account, AEC members serving on multiple AECs only need a single account, etc. This improves usability for them. We are still able to remove their account once both of the restricted organizations they are in are deleted/archived.
  • All users can authenticate via any known and supported mechanism, such as CILogon, ORCid, social-sign ons, or username/password of their choosing
  • Org maintainers can still debug restricted account issues by vectoring through those accounts as described above
  • We no longer need to auto-generate credentials and e-mail them in plaintext to class users.

The downsides of this approach:

  • Demos/classes require users to register accounts on the portal. Invitation codes with open enrollment windows can facilitate
  • This requires some (hopefully minor) updates to the EDU interface and workflow
19

An extension of this proposal:

All users in an organization can (optionally) be assigned an organization ID that is named according to the existing Deter convention. For example, bkocolos could be assigned ID org123aa when he joins org123. These IDs could be used as aliases for the identity on exp nodes; e.g., my accounts on materializations would be created as /home/org123aa. We could symlink between this and the actual username in mergefs, etc.