Orchestration advice (space swapping and package repointing)
Looking for some quick advice on orchestration.
If I have the following set of spaces for each environment where consumer points to packages in the models and dimensions, models point to packages in dimensions and sources, and dimensions points to packages in sources…
- TEST-Consumer
- TEST-Model X
- TEST-Model N
- TEST-Dimensions
- TEST-Sources
- ~~~~~~~~~~~~~~~
- PROD-Consumer
- PROD-Model X
- PROD-Model N
- PROD-Dimensions
- PROD-Sources
…and I want to do a promotion of content from TEST to PROD in that all spaces model/data/reporting contents get swapped and everything is repointed correctly to the packages in its own environment.
What is the ideal minimal set of steps with swapspacecontents, swapspaceforpackages, repointpackages, etc. to accomplish this?
I'm just trying to avoid too much playing around since the docs are a bit unclear to me.
The initial thought is to do swapspaceforpackages on each space and then repoint each space - is there a better way?
Thanks!
Best Answers
-
Within the Application Management AutoGenerated Clone Workflow, two versions of the copyspace parameters are applied and can be used in your custom Orchestration.
For Model and Source spaces:
repository;data;catalog;connectors;settings-permissions;packages
For Consumer:
repository;data;catalog;tags;packages;conditions;connectors;color-palettes;custom-subject-areas;dashboardstyles;catalog:DB2Bookmarks;settings-permissions;CustomGeoMaps.xml;spacesettings.xml;SavedExpressions.xml;DrillMaps.xml;settings-basic;themes;translations
Notice 'data' for raw data source structures. If you want Birst BI/DW then add datastore.
You can't avoid downtime with migration. It is either catching up on the data using the daily data processing method after the migration. Or creating copies of the PROD spaces, applying the changes, catching up on the data, and then moving to the live PROD spaces.
0 -
The copyspace parameter 'data' (Copy/replicate raw data files) refers to the files within the tenant file structure (those structures created on extract/publish). These files are the raw data structures as seen in Prepare.
The copyspace parameter 'datastore' is the physical table in the Birst BI/DW. Below are parameter variations:
- datastore-warehouse: Copy/replicate the data warehouse tables (i.e. – DW*) and indices.
- datastore-warehouse_tables: Copy/replicate the data warehouse tables.
- datastore-warehouse_indices: Copy/replicate the data warehouse indices.
- datastore-staging: Copy/replicate the data warehouse staging tables (i.e. – ST*) and indices.
- datastore-staging_tables: Copy/replicate the data warehouse staging tables.
- datastore-staging_indices: Copy/replicate the data warehouse staging indices.
- datastore-aggregates: Copy/replicate aggregates.
Yes, it is advisable to do both for a full data copy. If the data being loaded into TEST is production-ready data (not non-prod, sample), then the parameters can be used.
1 -
Got it, thanks again Diane!
0
Answers
-
EDITED with updated PPT with screenshots to clarify walk-thru section
If your set of spaces is an Analytic Solution, I'd recommend using the Application Manager functionality to clone PROD to preserve it as a backup. See the attached presentation for information on its use.
My preference is to use copyspace than swapspace as there is more parameter control as to what to migrate across spaces. Also, you can review the Auto-Generated Application Orchestration created by the Application Management clone to create your reusable Orchestration with copyspace, repoint packages, and if needed repoint connections.
0 -
Hi Diane. For some reason I can't open that file which is strange.
In either case, this is not for a prebuilt analytics solution, it's a custom set of spaces.
My process of going from DEV to TEST is basically that, using copyspace, then repointing. Typically, my copyspace command includes the following: replicate connectors;repository;catalog:"shared";SavedExpressions.xml;conditions;color-palettes;dashboardstyles;custom-subject-areas
I know doing it this way requires some downtime after the copy to reprocess the changes in the model, update the data, etc.
If I am using copyspace, is there a way around that, or additional params I should use so it's a seamless, no downtime transition to prod?
0 -
I appreciate the insight, thanks Diane. Avoiding downtime was a reason I've used swapspace previously for single space structure, though it sounds like with data copying we might be able to achieve that similarly in this structure, I.E. processing the data in all TEST spaces, then copying all spaces with their updated data to PROD? In that case, could you possibly elaborate on data vs datastore? I'm not sure I understand what you mean by raw data source vs BI/DW. Assuming I would need to do both if I want a full data copy?
0 -
Dear @Diane Mcelhiney,
I guess these Powerpoint slides have been explained extensively in some live presentation. If so, is there any recording?
(In particular: what does "CSDA" mean?)
0 -
@Enrico Menotti I have enhanced the PPT with screenshots for walk-thru section. CSDA = CSD Analytic Solution.
1
