Skip to content

feat: Adds support for multitenancy #555

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 190 commits into from
Jun 2, 2023
Merged

feat: Adds support for multitenancy #555

merged 190 commits into from
Jun 2, 2023

Conversation

rishabhpoddar
Copy link
Contributor

@rishabhpoddar rishabhpoddar commented Jan 13, 2023 •
edited by sattvikc
Loading

Summary of change

Adds ability to load from different config values based on tenant id and connection uri in requests. This also enables the user to use different connection pools per tenant.

Related issues

Test Plan

  • get resource from resource distributor based on precedence of checking:
    • most specific first
    • connection url based
    • tenant id based
    • null, null based
    • if most specific doesn't have a key, then we go to the (connection, null) setting, and then to the (null, tenantId) setting and then to the (null, null) setting.
  • If ee flag doesn't have multi tenancy, then Config.loadAllTenantConfig is not called.
  • Config tenant loading (should also merge plugin configs)
    • Non multi tenant continues to work
    • Multi tenant base config continues to work
    • Multi tenant specific to base merges correctly.
    • Synchronisation done on the core config is right.
    • Calling the loadAllTenantConfig function with different stuff saved in the db has effect on the loaded config - DONE
    • Calling getConfig with different connectionUri and tenantId provides the right config based on the setting
    • Invalid config across jsons being merged should throw an error when calling loadAllTenantConfig
    • Conflicting configs test for db plugin as well per tenant. Similar to mergingTenantWithBaseConfigWithConflictingConfigsThrowsError one
    • Test all conflicting and non conflicting configs
    • Test how a userpoolid is defined.
    • Test different API keys being used across different tenants.
    • Test different ip allow / disallow regex being used across different tenants.
    • Test that mistyped config in json will throw an error - DONE
  • ResourceDistributor's clearAllResourcesWithResourceKey function works. - DONE
  • Adding 100k tenants works fine in terms of the time and resources taking for checking the validity of their config.
  • User pool ID vs connection pool ID tests
  • Test new version of getTableSchema
  • Cronjob tests
    • adds two tenants with same user pool id, yields only one cronjob
    • adds two tenants with different user pool id yields two cronjobs
    • Add test to ensure number of cronjobs don't increase with new tenants - DONE
  • Test getting connection uri domain from actual request works well:
    • https without port yields 443
    • http without port yields 80
    • sub domain is also revealed
  • Storage layer that is removed (cause tenant is removed), closes connection pool and logging
  • Add test for NotFoundOrHelloAPI with base path set. - DONE
  • TenantnotFound exception tests
  • When adding a new tenant using a connection uri that is not associated with an app, then it should add that tenant to the null connection uri app - DONE
  • StorageLayer for tenants that are removed should also be removed when the tenants list is refreshed - DONE
  • AppId tests (similar to pathrouter test)
  • Multitenancy and MultitenancyHelper CRUD - DONE
  • emailpassword + multitenancy
  • useridmapping + multitenancy
  • Even if many apps and tenanats are there for the same user pool ID, the size list in cronjobs is still one, even after refreshing the list. - DONE
  • Adding core specific config (like argon2_hashing_pool_size should not be allowed via create tenant API) - DONE
  • deletion of user which doens't belong to the right tenant should fail - DONE
    • with user id mapping - DONE
    • without user id mapping - DONE
  • refresh and access token created using tenantId should contain the tenantId, unless the tenantId is the default one. - DONE
  • checking tenantId from created session - DONE
  • using wrong app should cause session refresh / verification to fail with unauthorized error - DONE
  • deleting an app deletes its info from ALL tables - DONE
  • deleting a tenant deletes tenant specific info from all tables, but keeps the user_id intact (for example in email password users table) - DONE
  • extracting the app id and tenant id works even if used docker service name with an _
  • if an app is removed, then the eelicensecheck cronjob for that app is not removed - we need to remove that as well.
  • Tests for new cronjob functions
  • Whenever a user is disassociated with a tenant, we need to also clear the user_roles table for that user (and other non auth recipe tables). We also need to write a test which will loop through all non auth recipes, and add info for a user for each of the non auth recipes per tenant, and then when the user and tenant mapping is gone, those tables should also be cleared. Similar to this test: https://github.com/supertokens/supertokens-core/blob/master/src/test/java/io/supertokens/test/userIdMapping/UserIdMappingTest.java#L778 - DONE
  • "TODO: add recipe usage + RAM tests + optimise how these 1000 tenants are created" -> testCreating1000TenantsWithOneStorageUsage (uncomment the test)
  • "TODO: we need to test recipe usage for the apps + RAM usage." -> testCreating50StorageLayersUsage (in plugin tests)
  • Ensure no TestOnly functions are used in production code
  • Check if any extra validations are required for gitlab and bitbucket providers while updating the SDK
  • Test cases in postgresql/SuperTokensSaaSSecretTest.java (see TODOs) - DONE
  • Add test for config for various third party providers (including using skipValidation) - DONE
  • Add test to check deleting app or tenant clears other dependent tables (using addInfoToNonAuthRecipesBasedOnUserId) - DONE
  • Config tenant loading (should also merge plugin configs)
    • Conflicting configs test for db plugin as well per tenant. Similar to mergingTenantWithBaseConfigWithConflictingConfigsThrowsError one
    • Test all conflicting and non conflicting configs
    • Test getting connection uri domain from actual request works well: - test the function
    • https without port yields 443
    • http without port yields 80
    • sub domain is also revealed
  • Storage layer that is removed (cause tenant is removed), closes connection pool and logging - test using number of threads - Connection Pool Closing DONE
  • TenantnotFound exception tests
  • StorageLayer for tenants that are removed should also be removed when the tenants list is refreshed - should not exist after refresh
  • AppId tests (similar to pathrouter test) - for different values - DONE
  • Add tenantid, appid validation
  • deleting an app deletes its info from ALL tables - DONE
  • extracting the app id and tenant id works even if used docker service name with an _
  • Tests for new cronjob functions
  • "TODO: add recipe usage + RAM tests + optimise how these 1000 tenants are created" -> testCreating1000TenantsWithOneStorageUsage (uncomment the test)
  • "TODO: we need to test recipe usage for the apps + RAM usage." -> testCreating50StorageLayersUsage (in plugin tests)
  • Check if any extra validations are required for gitlab and butbucket providers while updating the SDK
  • for nodejs, even with the latest CDI (3.0), without any changes to the sdk, all existing tests should pass

Documentation changes

TODO

Checklist for important updates

  • Changelog has been updated
    • If there are any db schema changes, mention those changes clearly
  • coreDriverInterfaceSupported.json file has been updated (if needed)
  • pluginInterfaceSupported.json file has been updated (if needed)
  • Changes to the version if needed
    • In build.gradle
  • If added a new paid feature, edit the getPaidFeatureStats function in FeatureFlag.java file
  • Had installed and ran the pre-commit hook
  • If there are new dependencies that have been added in build.gradle, please make sure to add them
    in implementationDependencies.json.
  • Issue this PR against the latest non released version branch.
    • To know which one it is, run find the latest released tag (git tag) in the format vX.Y.Z, and then find the
      latest branch (git branch --all) whose X.Y is greater than the latest released tag.
    • If no such branch exists, then create one from the latest released branch.

Tracking recipe changes

  • Auth Recipe / General Queries
    • Update APIs to use the right function - getTenantIdentifierWithStorage or getAppIdentifierWithStorage
    • Update to use storage function from TenantIdentifierWithStorage or AppIdentifierWithStorage
    • Schema and query changes
    • Multitenancy tests
  • Dashboard
    • Update APIs to use the right function - getTenantIdentifierWithStorage or getAppIdentifierWithStorage
    • Update to use storage function from TenantIdentifierWithStorage or AppIdentifierWithStorage
    • Schema and query changes
    • Multitenancy tests
  • TOTP
    • Update APIs to use the right function - getTenantIdentifierWithStorage or getAppIdentifierWithStorage
    • Update to use storage function from TenantIdentifierWithStorage or AppIdentifierWithStorage
    • Schema and query changes
    • Multitenancy tests
  • Email Password
    • Update APIs to use the right function - getTenantIdentifierWithStorage or getAppIdentifierWithStorage
    • Update to use storage function from TenantIdentifierWithStorage or AppIdentifierWithStorage
    • Schema and query changes
    • Multitenancy tests
  • Email Verification
    • Update APIs to use the right function - getTenantIdentifierWithStorage or getAppIdentifierWithStorage
    • Update to use storage function from TenantIdentifierWithStorage or AppIdentifierWithStorage
    • Schema and query changes
    • Multitenancy tests
  • JWT
    • Update APIs to use the right function - getTenantIdentifierWithStorage or getAppIdentifierWithStorage
    • Update to use storage function from TenantIdentifierWithStorage or AppIdentifierWithStorage
    • Schema and query changes
    • Multitenancy tests
  • Multitenancy
    • Update APIs to use the right function - getTenantIdentifierWithStorage or getAppIdentifierWithStorage
    • Update to use storage function from TenantIdentifierWithStorage or AppIdentifierWithStorage
    • Schema and query changes
    • Multitenancy tests
  • Passwordless
    • Update APIs to use the right function - getTenantIdentifierWithStorage or getAppIdentifierWithStorage
    • Update to use storage function from TenantIdentifierWithStorage or AppIdentifierWithStorage
    • Schema and query changes
    • Multitenancy tests
      • Same user (with same user id) added to 2 different tenants. Create code on both the tenants, then delete the user app wide, and check that consume code does not work on both the tenants as the device for that email or phone number should be deleted.
  • Session
    • Update APIs to use the right function - getTenantIdentifierWithStorage or getAppIdentifierWithStorage
    • Update to use storage function from TenantIdentifierWithStorage or AppIdentifierWithStorage
    • Schema and query changes
    • Multitenancy tests
  • ThirdParty
    • Update APIs to use the right function - getTenantIdentifierWithStorage or getAppIdentifierWithStorage
    • Update to use storage function from TenantIdentifierWithStorage or AppIdentifierWithStorage
    • Schema and query changes
    • Multitenancy tests
  • Useridmapping
    • Update APIs to use the right function - getTenantIdentifierWithStorage or getAppIdentifierWithStorage
    • Update to use storage function from TenantIdentifierWithStorage or AppIdentifierWithStorage
    • Schema and query changes
    • Multitenancy tests
  • Usermetadata
    • Update APIs to use the right function - getTenantIdentifierWithStorage or getAppIdentifierWithStorage
    • Update to use storage function from TenantIdentifierWithStorage or AppIdentifierWithStorage
    • Schema and query changes
    • Multitenancy tests
  • Userroles
    • Update APIs to use the right function - getTenantIdentifierWithStorage or getAppIdentifierWithStorage
    • Update to use storage function from TenantIdentifierWithStorage or AppIdentifierWithStorage
    • Schema and query changes
    • Multitenancy tests

Remaining TODOs for this PR

  • Remove deprecated init, getInstance and getConfig functions
  • Add tenantId to logging
  • Change resource distributor for plugins too? (I don’t think this is needed.. but maybe?�)
  • enable feature only if ee feature flag has right feature enabled. (only in add / update third party config / providers API)
  • load all tenant configs based on tenant config map in db:
    • Need to make sure that variables that can't be scoped based on tenantId in the same db don't have conflicting values.
  • add this for all tenants based on connection url + db separation only (and not tenant id)
  • feature flag usage stats - what needs to change here?
  • start as many instances of the cronjob as required based on number of tenants. It might be per db or per db & some config based on what the cronjob needs.
  • password hashing pools
  • Changes to the hello API?
  • SaaS changes
    • Deploying a new shared core per version of the core
    • Migration of current users and freeing up ec2 instance usage
    • Limiting how tenants who have not started a supertokens subscription will have only non ee features even though the license key is added to the cores we run.
    • Allow people to create their own connectionUri
  • Allow enabling of email password and passwordless per tenant?
  • getInstance of Config.java should run the fallback algorithm to returns values from most specific selection to least.
  • Implement assertAllTenantConfigs in config.java
  • Add usage stats for ee stuff:
    • Type of login and tenant detail as well. Not just that the tenant exists. Cause we may want to have different pricing for different login methods.
  • Transferring tenants - allow changing tenant of a user.
  • Change plugins config validation to not throw quit program exception, and instead throw an invalid config exception
  • getNormalisedConfigsForAllTenants should not take Config as baseConfig, but should take json representation of it cause it also needs to merge keys of plugin.
  • In APi to set config for tenant, we do not allow null, null to be set in the db - cause that refers to the config.yaml file setting
  • Confirm if the logic for "assertAllTenantConfigsAreValid" is correct?
  • Should we change getUserPoolId to read from the database and return an ID as opposed to just rely on the connection info? - We discussed about taking userPoolId from the user, but we can do it later.
  • Disallow changing of certain core configs per tenant (like port, host...)
  • core API key, ip allow / deny checks per tenant config based on incoming request
  • Checkout console output on terminal on core start with multi tenants (config checking should not have any extra logs)
  • Remove the use of deprecated setResource in resource distributor
  • We need to get rid of QuitProgramFromPluginException cause if one tenant has an issue, then the whole core shouldn't stop.
  • Memory leak debugging test
  • Change how unique pool and connection ID are created to hash the output
    • cause the user may type in a huge URL and then it may cause issues in our string maybe - especially where we set the connection pool name in hikari
    • decided not to do this as of now, as there are no limits is documented anywhere
  • implementation and use of clearAllResourcesForTenant when a tenant's info is updated (in resource distributor)
  • postgresql in CICD - cause some tests are only written with postgresql
  • confirm that this is OK - if (c1, t1) is set, and we query for (c1, null) or (null, t1), then it returns the result of (null, null) - add test for normalised config
  • Is it OK to run DeleteExpiredAccessTokenSigningKeys once every 24 hours?
  • Argon2 password hashing pool size increase cronjob for our SaaS -> every 5 users who say they wan argon2, we increase the pool size by 5 (and restart the core)
  • In docs:
    • need to change the host to 0.0.0.0 so that it can listen on multiple hosts
  • SAML server deployment?
  • SaaS changes:
    • healthcheck cronjob
    • Change in MAU calculation.
    • how a new app is created
    • Affect of telemetry and analytics - since we are only sending one telemetry ping per primary db.
    • How will users be allowed to configure a different db instance? And disallow them to connect to their own db instance outside of our SaaS?
    • Apply compute saving plan post multi tenant (see in aws cost explorer - recommendations section)
  • Dashboard APIs should be per app only
  • Tenant ID should be part of the path of the request
  • JWKs and hello API special case where we need to return a union of tenants:
  • Make APIKey per app and not per tenant because in the backend SDKs, keys are with respect to an app.
  • What should we do about handshake API?
  • When creating a tenant, we should treat deaultTenantId as null tenantId
  • TenantId format criteria:
    • case insensitive
    • can only contain alpha numeric chars along with hyphens
    • strings like "recipe" should not be allowed
    • Should not start with appid-
  • Make sure that logs from WebServerAPI.service work fine + add connection uri domain logging as well to it.
  • Should we allow creation of tenants based on connectionURI only if (connectionUri, null) tenant also exists?
    • change in getInstance in resource distributor
    • change in doesTenantExist function in resource distributor
    • change in config normalisation
    • change to not use quitprogramexception if resource not found - instead we should throw tenenatNotFound exception
  • Cronjob to refresh tenants in the core very frequently - this is useful when many cores are deployed and a new tenant is created.
    • In our saas, maybe we can detect this and call the API to refresh tenant list from the core.
  • When calculating pricing, we should treat a tenant as unique (connectionuri, tenantId). We should also count a tenant as in use when it has at least one user / session cause there may be a case where defaulttenantid exists for a specific connectionuri which has no user cause all the actual tenants in the app are t1, t2 etc. This is possible if the dev needs to create a new app before adding a new tenant.
  • How to solve the problem of using multiple cores behind a load balancer?
    • User creates a new app which is done via one of the cores and only that core now knows about this new app
    • User immediately queries again to create a new tenant for that app. This time, it hits another core and that core doesn't know about this new app (yet), defaulting to the null app.
    • This can be solved by resyncing with the list of tenants in the db in the getResource function before trying the null connection uri.
      • But it can lead to situations wherein someone malicious can query someone else's core via their own DNS with different connection uris and this will cause the core to keep on hitting the db.
      • This malicious situation is true even if the api key is wrong, cause even fetching the API key requires calling the getResource function.
  • In a kubernetes / docker-compose scenario, the connection uri is a service name - will the core see that?
    • If not, then this will cause issues with identifying the app cause:
      • The IP address might keep changing based on docker's network
      • If the dev wants to query the core via postman, they will probably use localhost which doesn't match the IP in a docker network.
    • If yes, then how will dynamic service names be generated if a new app wants to be added dynamically?
      • If the dev wants to query the core via postman, they will use localhost and not the docker service name anyway.
    • Another issue is that devs might end up creating a new app by mistake using 127.0.0.1 instead of localhost without realising it, and then later use localhost by mistake and wonder whats happening.
    • If the core is running on localmachine, and the dev wants to create many apps, they can't just do that easily. They will have to have different DNS settings..
    • This can be solved by not using the domain as the source of app name, or using that and also something else as the source of the app name... but what - and how will the core know which app name to use?
      • <connection uri>/appId?/tenantId?/path (appId can be part of connectionUri configured in supertokens.init + works well with JWKs endpoint config.)
      • <connection uri/appid-<appId>/tenantId?/path (appId can be part of connectionUri configured in supertokens.init + works well with JWKs endpoint config.)
      • <connection uri>/appId?/path?tenantId=... (appId can be part of connectionUri configured in supertokens.init - does jwks endpoint allow for query param?)
      • <connection uri>/path?tenantId=...&appId=...
        • requires change in sdk to add appId as part of backend supertokens.init
        • change in SDK querier to add this to all requests made.
        • change in supertokens.com dashboard to display appId
        • does jwks endpoint allow for query param?
      • make a tenant (connectionuri, appId, tenantId) instead of just (connectionuri, tenantId)
      • These changes also imply that resource distributor would no longer fallback on null appId cause it's being explicitly added by the dev in the API call being made to the core.
      • If we do not allow to identify an app based on connection uri, then existing SaaS users migration will be very difficult as users will have to modify their backend config.
  • change connectionuridomain to appid everywhere, so it will be (appid, tenantid)
  • Add AppNotFoundException vs TenantNotFoundException
  • Change default tenant id to "public".
  • For user roles
    • if a user wants to create one set of roles in the base tenant and reuse them for all tenants' users, how can they do that?
    • if the dev wants to have a common pool of users and roles across apps and assign them to users based on their tenantId, how will that work?
  • Add tenantId column to all tables and make it have foreign constraint on the main tenantid table (for that db)
  • When creating an appId, should we enforce different API key to be used? This prevents dev errors where they might put an appId of another app by mistake
  • Deleting tenant should delete all its info + deleting appId should delete all its info
  • What happens if license key expires in the middle whilst the core is running and many tenants are in memory?
    • what happens when the core with tenants is starting without license key
  • How do we count paid tenants if someone creates and deletes a tenant within a month? A situation can like this arise if someone is create different dev envs on the fly
  • Allow one user to be in many tenantId -> as long as all these tenantIds are in the same userpool (same db).
  • Allow roles to be associated with many tenants -> as long as all these tenantIds are in the same userpool (same db).
  • Should the user have to explicitly set a userPoolId instead of us implicitly getting it from the db connection string?
    • what happens if we don't do that, but the connection string changes
      • in an inconsistent way
      • in a consistent way (previous connection uris all point to a new, but the same connection uri) - all pointing to same db
  • on core start, create the default tenant - with all login methods enabled + empty providers list
  • Should the core check for which recipes are enabled before allowing that tenant to use that recipe's functions? Or should this only be left to the backend SDK?
  • Confirm that if license key stops, then we still allow the tenant pool to continue to work. Adding new tenants or cross association of users across tenants won't work - test
    • We should perhaps disable creation of users in non default tenant as well?
  • Need a cronjob to refresh the tenant list in memory - cause there are a few operations which will read from that list during API calls (like sign up email password, if the tenant has enabled that or not). And if the config is changed in another core, it should eventually reflect in other cores as well. -> Not needed because we call refresh list in resource distributor
  • See TODO in loadAllTenantStorage:
    • about moving the close and init of storage to outside of the synchronised block
    • about having to readd the base tenant at the end (is that needed cause the input always contains the base tenant)
  • Should we split resource distributor to have a different map per connection uri and appId and do locking based on that value as opposed to applying a global lock?
  • Check equality of Provider in ThirdPartyConfig works fine, especially when additionalInfo has sub objects in it
  • Implement ThirdParty.verifyThirdPartyProvidersArray
  • TODO in getUsersCount and getUsers function for if includeAllTenants
  • Check user pagination function logic when doing per tenant or across all tenants
  • Recheck core constraint that can be different across tenant IDs + make sure that check that they can be different across app IDs.
  • if new connectionuridomain, then make sure that user pool id is unique
  • Whats the effect of running cronjobs in parallel across tenants - might cause running out of db connections cause the same db connection pool is shared across all those tenants.
    • We run cronjobs in parallel per storage object - so this should not be a problem.
  • User object returns an array of tenants
  • removeUserIdFromTenant in multitenancy
  • removeRoleFromTenant in multitenancy
  • How will deleteUser work? If the user is shared across tenants - we should ideally remove that user across all the tenants.
  • Solve for "some configs must be same app wide (like access token signing key update)." in assertAllTenantConfigsAreValid
  • Do not allow multi tenancy in case of nosql being used. But session for base tenant should continue to work with mongodb
  • Telemetry API change - to make it per app?
  • Docs - need to mention that creation of roles happens on a db level, so all tenants in that db for that app can use the role. But if there is a tenant in another db for that app, then that tenant cannot reuse this role and will need to be recreated for them.
  • createNewRoleOrModifyItsPermissions function should also add to the tenantId <-> role table -> roles are per app now
  • Add tenantId to the access token and refresh token which is added during create new session and used during getSession / refresh session functions in the core - so the APIs do not take a tenant ID here, but just the appId
  • Do not allow api key change across tenants, but only across apps
  • We need to add tenantId to the reset password link as a query param in the backend SDK
  • We need to add tenantId to the email verification link as a query param in the backend SDK
  • We need to add tenantId to the password reset link as a query param in the backend SDK
  • There are some functions in emailpassword and other recipes which only really depend on the appIdentifier, like getUserUsingId. But we still ask the user to use the right tenantIdentifier cause we want to identify the right user pool - should this change such that we find the right user pool on our own?
  • The core configs stored in the db should not be normalised - only whats provided by the user.
  • We need to disallow the user to set certain core configs dynamically
  • Change license key to be per app, and the API ping for it should also send the app info associated with that license key (both connectionuridomain and appId). This also means that the feature flag thing + ee cronjob will have to change.
  • Change getUsers API to also have includeAll query param which will return results across all tenants for the app
    • We decided not to do this cause it is possible for 2 tenants to be in different databases
  • In usage stats, send the whole tree of connectionUris -> apps -> tenant.
  • Validate core config specified for tenant to throw an error if an unknown config is specified.
  • In config.yaml (for core and plugins) + devConfig.yaml, we should add a comment against each config saying DIFFERENT_ACROSS_TENANTS or DIFFERENT_ACROSS_APPS. since in the docs, we say to lookout for this comment to know which config can be different per tenant.
  • getPaidFeatureStats in ee feature needs to properly get all the apps, and not just the base app
  • Add a doTask function in CronTask per app as well - so that cronjobs like the telemetry one do not send multiple pings for the same app concurrently.
  • if an app is removed, then the eelicensecheck cronjob for that app is not removed - we need to remove that as well.
  • recheck permission of multi tenancy operations and tally them with saas changes
  • Resolve "// TODO ignore only for now, this function should throw this exception" in WebserverAPI.java file
  • The waiting for db connection to happen on core start should only happen for the base tenant. That's cause users may create an app / tenant with a random db connection uri which doesn't exist, and then it should not have any side effect other than when actually using that app - it should throw a 500 error from the API (since the db doesn't exist).
  • What are the new APIs for tenants and multi tenancy in general? For example - does user exist in a tenant?
  • Do a global search on where all getUserIdMapping function is being used and review if it needs to be used in that place or not. It should never be used where we have a user ID as an input in the API, or in the functional layer?
  • Need a way to know how many apps and tenants were created for the billing cycle even though those apps / tenants may not exist at the time of checking them.
  • Creating many dbs in cicd (see comment below for node script)
  • Older CDI versions should ignore appId and tenantId (should treat both as public) - but not CUD (cause of SaaS)
  • Should we re add the base tenant resource in the AccessTokenSigningKey.loadForAllTenants? Cause the apps input will always have the baseTenant. Also, for some weird reason, if we do not add the base tenant, and we do not call AccessTokenSigningKey.initForBaseTenant(this); in main.java, the session tests still pass.. not sure why.
  • Add index on expires_at in session_info table
  • for multi tenancy related APIs, the feature flag check should be done on the null, null, null tenant and not the app level for whom the request is coming from.
  • Add a new env for supertokens_saas_secret.
    • For any API that returns core config (related to database config), should strip the db configs, unless the supertokens_saas_secret is defined, and is being used.
    • For any API that modifies / creates a new core config (related to database config), we should not allow db related configs, unless the supertokens_saas_secret is defined, and is being used.
    • We do not show / modify:
      • postgresql_connection_pool_size
      • postgresql_connection_uri
      • postgresql_host
      • postgresql_port
      • postgresql_user
      • postgresql_password
      • postgresql_database_name
      • postgresql_table_schema
  • Change /config API to only be queried by the base tenant
  • In all APIs that return the user object (like sign in, get user by ID, get user by email etc..), we need to add the tenantIds
  • Make a migration script for the db
  • The shouldInitialiseJWTSigningKeys boolean should be used alongside Main.isTesting in the actual prod code.
  • Check what happens if you change the access token update interval for a tenant dynamically.. what other config like this exists -> what happens if you change the storage layer of a tenant dynamically? Or some other config?
  • See why psql logs are not showing up in info / error.log when core start using java command.
  • When removing a user id from a tenant, we also need to remove from all non auth recipes (needs to be in sync with delete user and also be easy to maintain).
  • Update contributing guide for dbs to mention that we need to create multiple databases and give which ones and how many exactly.
  • backend sdk should protected prop check for tenantId in the accesstoken payload
  • Pass tenantId into sendEmail and sendSms functions on the backend SDK as well.
  • confirm is that if we pass the test_mode flag to the core in the java command, it has the same effect as setting the isTesting to true (i think)
  • Add Microsoft login (not AD) to the backend SDK
  • Update recipe list on supertokens.com dashboard to allow users to choose multiple recipes in case they want to do that for different tenants. Right now, it's a drop down, so it doesn't really work for this case.
  • Create new version of access token with tId
    • Old access token with Old CDI
      • Should verify accesstoken as is without the need for tId
      • Should refresh accesstoken without tId, and keep the old accesstoken version
    • Old access token with New CDI
      • Should verify accesstoken as is without the need for tId
      • When refreshed, should create new version of access token with tId
  • New CUD creation should be disabled with in mem db
  • Disable tests that use different user pools
  • Run e2e auth-react, website and node sdk tests with new core.
  • Postgresql needs to have explicit indexes created for all the foreign keys that it has.
  • testing with dashboard before releasing core
  • remove multitenancy claim validator in the backend SDKs

@rishabhpoddar rishabhpoddar changed the base branch from master to 4.3 January 13, 2023 10:44
@rishabhpoddar rishabhpoddar marked this pull request as draft January 13, 2023 10:44
@rishabhpoddar
removes unnecessary file
10afc48
@rishabhpoddar
adds a bunch of todos
65012d9
@rishabhpoddar
adds storage layer function and also laod all configs function
737563e
@rishabhpoddar
implements core config merging across tenants
19aade1
@rishabhpoddar
restructures code related to config checking in core
7ed1b8c
@rishabhpoddar
changes storage layer to take json instead of config file path
5723ba1
@rishabhpoddar
small refactor
3ef6397
@rishabhpoddar
adds ability to create new storage instances on the fly
650cffe
@rishabhpoddar
in mem db function impl for user pool ID and config conflict checking
1af722e
@rishabhpoddar
small change
b552ff8
@rishabhpoddar
adds checks for conflicting core configs
67382e5
@rishabhpoddar
adds a few tests for multi tenant config
76d7492
@rishabhpoddar
adds more tests
2ac6616
@rishabhpoddar
more tests
1b0d182
@rishabhpoddar
adds function to get connection pool ID
ba2923b
@rishabhpoddar
adds skeleton for loading storage layers for tenants
40ad561
@rishabhpoddar
fixes compile error
cb6e6d8
@rishabhpoddar
adds code to load multiple storage layers as well
579056b
@rishabhpoddar
deletes info across all tenants during testing
7c37142
@rishabhpoddar
adds one test for storage layer
392a8b0
@rishabhpoddar
fixes a test
1f35011
@rishabhpoddar
fixes bug in setting resource in resource distributor
079ed01
@rishabhpoddar
fixes bugs
c69ca66
@rishabhpoddar
loads signing keys for all tenants
5c2000d
@rishabhpoddar
fixes bug
0fcaf35
@rishabhpoddar
adds new test and modifies the testing kill process to clear database…
980f6c3 
…s across mulitple user pools
@rishabhpoddar
implements more test
2280002
This was referenced May 15, 2023
Merged
Merged
Merged
@sattvikc sattvikc mentioned this pull request May 18, 2023
Merged
11 tasks
sattvikc and others added 4 commits May 22, 2023 10:08
@sattvikc
fix: session v4 (#683)
a01f0f0 
* fix: session v4

* fix: tests

* fix: pr comments and tests

* fix: test

* fix: pr comments
@sattvikc
fix: Postgres migration (#685)
83ea154 
* fix: changelog

* fix: changelog

* fix: pr comments
@sattvikc
fix: fixed tests for active user storage (#686)
dcf11c2
@nkshah2
refactor: Add email to the response of Dashboard verify session API (#…
442e83c 
…689)

* Update Dashboard verify session API to return the user's email

* Update tests

* Update CHANGELOG

* Make changes based on PR comments
@sattvikc sattvikc mentioned this pull request Jun 1, 2023
Merged
11 tasks
@sattvikc
fix: Inmemory plugin (#693)
c9f7fd8 
* fix: inmemory impl

* fix: test fixes

* fix: test fixes
@sattvikc sattvikc changed the title Feature: Multi tenant config feat: Adds support for multitenancy Jun 2, 2023
@sattvikc sattvikc changed the base branch from 4.3 to 5.0 June 2, 2023 09:18
sattvikc and others added 3 commits June 2, 2023 15:00
@sattvikc
fix: update version
91907e1
@sattvikc
fix: CDI version update
39c1505
@sattvikc @porcellus
fix: mongo plugin (#694)
6290290 
* fix: mongo plugin

* test: fix mongodb tests

* fix: pr comments

---------

Co-authored-by: Mihaly Lengyel <[email protected]>
@sattvikc sattvikc self-assigned this Jun 2, 2023
@sattvikc sattvikc marked this pull request as ready for review June 2, 2023 11:51
@rishabhpoddar rishabhpoddar changed the base branch from 5.0 to 6.0 June 2, 2023 12:27
@rishabhpoddar rishabhpoddar merged commit b294e0f into 6.0 Jun 2, 2023
@rishabhpoddar rishabhpoddar deleted the multi-tenant-config branch June 2, 2023 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@sattvikc sattvikc

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@rishabhpoddar @sattvikc @nkshah2