Publishing Issues
2025-01-15 20:17:02,761+0000 INFO - Finished ST_ICSEU_CS 2025-01-15 20:17:02,818+0000 INFO - Loading directly from script output - ICSEU_CS.txt.tmp 2025-01-15 20:17:02,828+0000 INFO - Bulk loading staging table ST_ICSEU_CS from source file C:/Birst1/Production/[CUSTOMER_PRD-SX-Source]/data/ICSEU_CS.txt.tmp 2025-01-15 20:17:03,670+0000 INFO - Successfully loaded ST_ICSEU_CS 2025-01-15 20:17:03,671+0000 INFO - Deleting ICSEU_CS.txt.tmp 2025-01-15 20:17:03,674+0000 INFO - Deleting ICSEU_CS.txt.tmp.format 2025-01-15 20:17:03,679+0000 INFO - Logging step LoadStaging [ST_ICSWU_CS] for processingGroup [400-IC_CS] for iteration 2, status Running 2025-01-15 20:17:03,683+0000 INFO - Starting LoadStaging [ST_ICSWU_CS] for processingGroup [400-IC_CS] 2025-01-15 20:17:03,692+0000 INFO - Starting ST_ICSWU_CS 2025-01-15 20:17:03,697+0000 INFO - Logging step LoadStaging [ACORN] for processingGroup [400-IC_CS] for iteration 2, status Running 2025-01-15 20:17:03,701+0000 INFO - Starting LoadStaging [ACORN] for processingGroup [400-IC_CS] 2025-01-15 20:17:03,709+0000 INFO - Staging table ST_icswu already loaded successfully for load id =2, for load group ACORN. This step will not be repeated. 2025-01-15 20:17:03,713+0000 INFO - Logging step LoadStaging [ACORN] for processingGroup [400-IC_CS] for iteration 2, status Running 2025-01-15 20:17:03,717+0000 INFO - Starting LoadStaging [ACORN] for processingGroup [400-IC_CS] 2025-01-15 20:17:03,720+0000 INFO - Staging table ST_txt__ICSWU already loaded successfully for load id =2, for load group ACORN. This step will not be repeated. 2025-01-15 20:55:28,127+0000 INFO - Processing script: ST_ICSWU_CS 2025-01-15 20:55:28,127+0000 INFO - Rows read from db in previous interval: 0, Total records read from db: 0, waiting for script execution: 0(Work queue capacity: 100), threads to execute script: 0(max: 1, active: 0), script execution completed: 0, waiting to be written to file: 0(Write Queue Capacity: 15189) 2025-01-15 20:55:28,134+0000 ERROR - Script halted. SQL Exception com.microsoft.sqlserver.jdbc.SQLServerException: An error occurred during the current command (Done status 0). Cannot continue the execution because the session is in the kill state. 2025-01-15 20:55:28,141+0000 ERROR - Script halted at record 0 com.successmetricsinc.transformation.ScriptException: Script halted. Bad query. Please examine your query and contact Birst support if needed. 2025-01-15 20:55:30,166+0000 ERROR - Exception (line: -1) Script halted at record 0 (Script halted. Bad query. Please examine your query and contact Birst support if needed.) com.successmetricsinc.transformation.ScriptException: Script halted at record 0 (Script halted. Bad query. Please examine your query and contact Birst support if needed.) 2025-01-15 20:55:30,169+0000 INFO - Logging step LoadStaging [ST_ICSWU_CS] for processingGroup [400-IC_CS] for iteration 2, status ScriptFailure 2025-01-15 20:55:30,177+0000 ERROR - Script halted at record 0 (Script halted. Bad query. Please examine your query and contact Birst support if needed.) java.lang.Exception: Script halted at record 0 (Script halted. Bad query. Please examine your query and contact Birst support if needed.) 2025-01-15 20:55:30,178+0000 INFO - Logging step LoadStaging [ACORN] for processingGroup [400-IC_CS] for iteration 2, status Failed 2025-01-15 20:55:30,185+0000 INFO - Elapsed Time = 40 minutes, 14 seconds for: LoadStaging 2025-01-15 20:55:30,188+0000 INFO - Elapsed Time = 0 minutes, 0 seconds for: ErrorContinuePoint 2025-01-15 20:55:30,198+0000 INFO - Logging step ExecuteScriptGroup [ACORN] for iteration 2, status Running 2025-01-15 20:55:30,201+0000 INFO - Starting ExecuteScriptGroup [ACORN] 2025-01-15 20:55:30,202+0000 WARN - Empty script group: There are no scripts defined in script group ACORN 2025-01-15 20:55:30,203+0000 INFO - Logging step ExecuteScriptGroup [ACORN] for iteration 2, status Complete 2025-01-15 20:55:30,209+0000 INFO - Finished ExecuteScriptGroup [ACORN] 2025-01-15 20:55:30,210+0000 INFO - Elapsed Time = 0 minutes, 0 seconds for: ExecuteScriptGroup 2025-01-15 20:55:30,213+0000 INFO - Elapsed Time = 0 minutes, 0 seconds for: ErrorContinuePoint 2025-01-15 20:55:30,215+0000 INFO - Logging step LoadMarker [ACORN] for iteration 2, status Complete 2025-01-15 20:55:30,219+0000 INFO - Finished LoadMarker [ACORN] 2025-01-15 20:55:30,220+0000 INFO - Elapsed Time = 0 minutes, 0 seconds for: LoadMarker 2025-01-15 20:55:30,246+0000 INFO - Elapsed Time = 0 minutes, 0 seconds for: Copy 2025-01-15 20:55:30,250+0000 INFO - Elapsed Time = 0 minutes, 0 seconds for: Touch 2025-01-15 20:55:30,257+0000 INFO - Elapsed Time = 0 minutes, 0 seconds for: Del 2025-01-15 20:55:30,263+0000 INFO - Elapsed Time = 0 minutes, 0 seconds for: Del
I am having an issue with the Initial Extract Workflow for CSD. I am hoping someone can help me out but probably need to go to support. I am able to see data in the Raw ICSWU table, but I am unable to see any data in the scripted ICSWU. When I try to execute the script, I end up getting the below screenshot. Seems to timeout at 5 minutes exactly and not give me any logs. The above log is from the publishing.log file.
Best Answer
-
The issue is the volume of all variations extracted for ICSWU is maxing temp space used in Birst DB for the ORDER BY. We see this situation with the use of the JDBC v1 Data Lake Connector (used in prior 2024.10 release of CSD Analytic Solution) because of the width of all the columns * the depth (volume) of the data result set.
The good news is there is an efficient way we can implement it. You need to enter a Support Case to the SX.e Technical Support team. We need to assess what version of CSD Analytics is in the environment (I'm assuming this is PRD publish?). If you have a version before 2024.10.02 then your environment will be added to the queue for an upgrade. This newer version uses the Infor Data Lake Compass driver to extract only current active variation. Therefore, as an example, if ICSWU in Data Lake contains 200 million rows of all variation, the active variations quantity could be 50 million rows. Time is decreased in extraction and publish within the Source space.
0
Answers
-
Diane - Thank you for the quick reply. Yes, this is a PRD Publish. That makes sense to me, but I am surprised that it extracts all variations. I think this customer is running v_2024.12.00.2245247, as that is what I see by the User Profile button. I don't know where else to look for version.
0 -
What you see under the User Name on the User Profile button is the version for Birst. What Diane is referring to is the version for CSD Analytics, which I believe is reported in the main dashboard - at least that's where it is in LN Analytics.
1 -
Yes, please switch to the SX-Consumer and in the top left-corner of the dashboard is the release.
1 -
It extracts all variations because when Analytic Solutions were introduced and connected to the Data Lake, the JDBC v1 driver was the only available connection. Each Analytic Solutions has been moving toward using the Compass driver provided in Birst to extract from the Data Lake.
1 -
Thank you @Enrico Menotti and @Diane Mcelhiney! They are running 2024.01 Version. Sounds like I need to open a ticket to find out when they will be updated.
0 -
Thank you for the explanation. I am surprised I didnt run into this more with some other customers. I am glad to know there is a solution right around the corner!
0 -
Yes, please submit the case directly to the SX.e Tech Support on behalf of the customer. If there is an immediate need, please email me and I can review a course of action in the meantime.
1 -
It depends on the customers Inventory Usage volume. I've see it for certain customers in particular industries. Also, there is a SAABE process which has also been implemented which decreases the data piped from the Cloudsuite to the Data Lake to complement the performance enhancement of using the Birst Compass driver.
Technical Support will assess these points when they prepare the Analytic Solution in Birst for upgrade.
0
