Solution

With the rapid development of IT applications, more and more users of finance, banking, government, etc. require that their core business is continuously connected to the Internet 7*24, and the continuous power supply will continue to operate. Then there has been a two-region three-center program. For natural disasters, choosing two computer rooms in the same city and choosing one computer room in different places is called two places and three centers. This solution has high availability and disaster backup capabilities. Multi-center emergency disaster simulation exercise program Simulation test

English

简体中文

Solution

Solution areas Industry solutions migrate Simulation test version upgrade Cloud centralized monitoring DB move down Disaster recovery backup Performance monitoring and diagnosis

Home

Products

0 series——fluoroscopy diagnostic series

X0: Database Perspective Cognitive System

1 series——put into production drill series

X1: Database simulation test systemm

A1: Application simulation test system

3 series——migration comparison series

T3: Incremental synchronization system

V3: Data comparison system

M3: Full migration system

cloud platform

Woxin Cloud goes to the cloud, sharing digital wealth
together. Use your professional abilities to provide
various digital services together through the platform .

learn more

Tweets

News Information

Company news Product dynamics

Special Column

Database migration Move host down

About +86 0755-85219803

What are the needs of users？

In combination with the large-scale natural disasters in China in recent years, the disaster recovery model of "two locations and three centers" with two centers in the same city and remote disaster recovery centers has also emerged. This solution has both high availability and disaster backup capabilities. However, there is currently no relevant tool to verify its function, so there is an urgent need for a tool that can realistically simulate business scenarios and drive disaster drills.

Understand the technical solution

How to quickly switch to the standby data center in case of unpredictable disasters？

Real business scenarios are simulated by simulation. The simulation reads the IP signal mapped by DNS every five seconds. When the signal changes, the data will be played back to the corresponding server by load balancing to ensure the normal transaction。

How to ensure that the overall business will not be affected in case of uncontrollable disasters？

The scheme of changing the old core from active and standby to two places and three centers. That is, the scheme of local double live + Remote asynchronous replication. The local data center is double active, and the standby data center provides real-time backup to the main data center to ensure that business processing will not be interrupted. Asynchronous data replication is adopted in remote data centers to reduce the probability of business data loss。

Solution advantage

Real business scenario Business traffic replay, transaction restoration, real scene reproduction；

One-click simulation One-click deployment, startup, and containerization of the entire simulation process greatly improve the effectiveness and operability of the exercise;

Horizontal expansion According to the needs of the exercise, it can achieve rapid expansion and deployment to achieve high-performance and high-quality exercise effects;

Performance and reliability verification Through pressurized playback, the application performance test of the three centers can be carried out;

Improve the efficiency of handover exercises Through automated simulation transaction playback, handover drills can be carried out at any time, reducing the preparatory work and personnel input of the drills;

Simple and efficient Convenient unified management platform, WEB visual interface, reduce operation and maintenance costs;

Real business scenario Business traffic replay, transaction restoration, real scene reproduction；

One-click simulation One-click deployment, startup, and containerization of the entire simulation process greatly improve the effectiveness and operability of the exercise;

Horizontal expansion According to the needs of the exercise, it can achieve rapid expansion and deployment to achieve high-performance and high-quality exercise effects;

Performance and reliability verification Through pressurized playback, the application performance test of the three centers can be carried out;

Simple and efficient Convenient unified management platform, WEB visual interface, reduce operation and maintenance costs;

First, we synchronize the data changes on the local standby machine to the AS400 migration machine through Mimix. Then, format analysis, transcoding, storage and sorting of the received as / 400 incremental logs are performed, and then the transaction window is identified simultaneously according to the data table to generate SQL, fill in the partition key according to the requirements of the distributed database, and send it to the distributed database. Finally, after receiving a single SQL statement sent by the synchronizer, the database agent completes the database write and update processing.

Checkpoint recovery

Various technical means such as file offset breakpoint, overall rollback of batch files and re-entry of memory file information are adopted to complete the breakpoint recovery of the whole link and ensure the correctness of data synchronous transmission.

Large field synchronization

All large field data is stripped from the log, and the mapping relationship between large field and row primary key and control field is constructed. The content is extracted and transcoded according to the meta structure of large field, recombined to form the text format supported by mysql, and then the update statement is generated. When the data without large field is applied, the apply action of large field UPDATE statement is triggered.

Table name heterogeneous policy

1) Table name replacement: solved by meta mapping relationship
2) One split multiple: one SQL statement in the original log needs to construct several different SQL statements according to the split rules
3) Multiple in one: the operation type of log records with primary key information remains unchanged, and log records without primary key information are converted to update statements.

Concurrent loading of primary key

The asynchronous connection mode of database client is adopted, and the queue is sorted according to the primary key value. Different primary keys can be loaded concurrently, and the operation is serial under the same primary key.

Data supplement

The comparison engine transcodes the log data in the error record set, supplements the fragment key, and generates the corresponding insert statement text file.

Migration guarantee Solve the problems of feasibility, reliability, safety and correctness of the move down scheme, and provide reliable technical guarantee.

Rapid migration The migration speed can reach 50g / h, shorten the shutdown switching window and greatly reduce the impact on the business.

No invasion There is no need to deploy programs in the production environment, and only read the standby database log, which has no impact on the production environment.

Flexible control The incremental migration process supports pause and restart. In case of failure, it can support breakpoint continuation.

Program self-healing The migration program has a daemon, which can heal itself in the whole process and reduce the difficulty of operation and maintenance.

No risk of data loss It can check the data migration after the data migration is completed, and verify the consistency of the data through a unique verification algorithm to ensure the integrity, correctness and effectiveness of the data.

Data migration is a very important link in the core system and needs to be completed in a very short window period. It faces the following pain points:

1. The source database object relationship and logical dependency between tables are complex
2. Target database architecture deployment, specification requirements planning
3. The business interruption time in the migration process is required to be short
4. Synchronous data comparison and verification is required
5. Reverse data backflow is required to ensure safety and apply the back cutting scheme; WoXin provides full link migration solutions from centralized database to distributed database: full migration, incremental migration, data comparison and verification, data supplementary recording / correction, which greatly reduces the threshold and improves the efficiency of data migration between heterogeneous databases.

The process of full data migration and full data comparison is as follows:

Multi concurrent (multi process) reads data files from the source table of the old core AS400, transcodes and generates standard data block files, downloads and transmits them to the x86 server, and then imports the data into the distributed database in batches according to the original table structure

Every time a table is migrated, a signal is sent to the comparison system, and the system drives the comparison component to quickly compare the field contents of the data table in the two sets of databases to complete the data comparison

Points consumed in this process:

1. The bandwidth performance of the network card is limited. The network card under AS400 is a Gigabit card
2. AS400 has a bottleneck in disk IO
3. During disassembly, some tables also need patch keys and large field conversion

In order to meet the requirements of data migration, one-time switching and real-time data comparison, incremental data migration and comparison are required:

1. The incremental data of as / 400 is synchronized to the distributed database in quasi real time.
2. Speed: complete the synchronization of 500g database incremental log of as / 400 core system within 12 hours.
3. Consistency: the data before and after data synchronization is required to be completely consistent.
4. Ease of use: support manual pause and start, errors encountered during synchronization, log recording and breakpoint continuation.

The incremental data migration process is:

400 local mirror: deploy local mirror in the local machine room of as / 400 core system to synchronize the data changes on the local standby machine through Mimix. Deploy jrn log acquisition program on the upper part of the mirror to collect incremental jrn logs in quasi real time and push them to the data synchronizer

Data synchronizer: after receiving the incremental jrn log of as / 400, it performs format analysis and transcoding, then stores and sorts it, identifies the transaction window according to the data table, generates SQL, fills the partition key according to the requirements of the distributed database, and sends it to the distributed database.

Distributed database: after receiving a single SQL statement sent by the synchronizer, the database agent completes the database write and update processing.

(incremental migration flowchart)

The incremental data comparison process is:

Analyze the historical information of the change record set of the change table in the jrn log, take the primary key information of the last change table, extract data from the target distributed database, and conduct real-time and rolling comparison with the relevant records of the target database.

(Incremental comparison process)

Overall one-time switching flow chart:

(X86 core data migration process plan)

Core switching product tool chain:

1.Full migration of heterogeneous complex tables

The as / 400 core system extracts the complex table, converts the code system, parses the large field type, converts the data content, supplements the data fragment key, and finally generates the insert statement and migrates the data to the distributed database in high concurrency and large quantities under the connection.

2.Full data comparison of heterogeneous databases

In the static state of as / 400 core system and x86 distributed database, quickly compare the field contents of the full data table in the two databases according to the specified data table range and field range.

3.Data log synchronization

Incrementally synchronize the database changes in the as / 400 core system to the distributed database to ensure that the data changes of the two databases are completely consistent. The data synchronization process supports pause and restart. In case of failure, it can support continuous transmission at breakpoints. The transaction consistency of the two databases needs to be ensured in the whole process.

4.Heterogeneous database sampling data comparison

Before system switching, the as / 400 core system and x86 distributed database continuously synchronize data, verify the consistency of data synchronization through non-stop rolling comparison, reduce the pressure of full data verification during system switching and reduce the switching time window.

Escort Solve the problems of feasibility, reliability, safety and correctness of the move down scheme and reliable technical guarantee to escort the switching.

Risk reduction Greatly reduce the switching risk and achieve the goal of one-time smooth switching and production.

Window shortening Shorten the expected shutdown switching window and reduce the business impact.

Lower migration threshold The details of distributed database migration are shielded for the operation and maintenance personnel to realize "zero error and zero loss" of data.

Self healing ability The whole abnormal switching recovery process does not need manual intervention, the whole process is self-healing and completed automatically, reducing the difficulty of operation and maintenance and management cost.

Horizontal expansion The capacity expansion of the comparison application can be realized through the user interface.

Working principle:

Through the network bypass image technology, the data is captured from the switch, and the analysis server of the perspective diagnosis system is responsible for analyzing the data and storing the analyzed data. The web page displays the quasi real-time data and alarm information returned by the web server

Database health status detection:

Database server and application server are regarded as nodes one by one, and they communicate with each other through TCP. By analyzing the data from the switch, the real-time running state of the application and database is observed.

The interaction between the application side and the database is established through request and response. The data between the application and the database is obtained through WoXin CNSTAT module, and the five states of hang, down, Britain, Asia and slow are obtained through analysis.

(1)HANG：

APP HANG：The application has a connection but no request or response for a period of time.

DB HANG：The database has connection and response rate over a period of time, but there is no response.

(2)Down：

APP Down：The application has no connection, no request and no response for a period of time.

DB Down：The database has no connection, no request and no response for a period of time.

(3)Bump：

Taking the average response time of a period of time as the data for linear fitting calculation, the variance of the average response time is obtained. The greater the variance, the greater the fluctuation of application / database average response time. If the variance exceeds the threshold, it is judged as high.

(4)inferior：

DB inferior：Over a period of time, the average response time shows an upward trend and the trend is greater than the threshold.

APP inferior：Over a period of time, the number of connections shows a downward trend and the trend is greater than the threshold.

(5)Slow：The average of application / database response time regression is greater than the threshold.

Advance prediction problem

Count the execution parameters of each type of SQL, analyze and obtain high concurrency hot spots and single point data, predict in advance whether there are hidden dangers, and put forward optimization solutions on this basis.

Cluster query SQL according to rules, count the times and average response time of different SQL statements and different query fields under the statement after clustering, and give early warning for those with long response time.

Fast location problem

4.1 Fault root cause analysis

abnormal sqlcode classification diagnosis: through the classification of sqlcode, directly give the relevant statements under the error connection, problem root cause, abnormal constant and the number of abnormal sqlcode types. For example, errors such as duplicate key values and full space can be explained directly.

4.2 TPS low diagnosis

Through the two dimensions of TPS and output rate, judge whether the responsibility lies in the database when TPS is low:

When TPS and output rate are lower than their respective thresholds at the same time, it can be determined that the responsibility is the database;

When the TPS is below the threshold but the output rate is normal, it can be determined that the responsibility is not the database.

4.3 Channel not submitted for diagnosis

It is found that the connection time exceeds the threshold, and there is still no commit / rollback application connection and the transaction. Find out the similar transactions through transaction clustering and predict the possibility of the next SQL.

4.4 Large transaction analysis and diagnosis

Discover and predict transactions that may become large transactions (the number of affected rows is greater than the threshold).

Analyze the SQL statements with the number of affected rows greater than the threshold, obtain the corresponding SQL type, and judge whether the proportion of the number of SQL statements with the number of affected rows greater than the threshold to the total execution times of this kind of SQL type reaches the alarm value. If the alarm value is reached, find the transactions that may become large transactions through the relationship between the transaction type and the SQL type.

4.5 Slow SQL statement type diagnosis

Find the SQL type whose average response time is greater than the average response time of the database.

Find the SQL type that most affects the database performance by the execution times of SQL type statements and the proportion of the total response time of this kind of SQL type in the total response time of the database.

Omnidirectional quasi real-time monitoring of database operation status

(1)Performance monitoring

[1] Output rate: the proportion of normal responses of the database to evaluate the service capacity of the database.

[2] Response rate: the proportion of database responses to requests, which is an important parameter to evaluate the service capacity of the database.

[3] Proportion of abnormal sqlcode: retrieve all transactions, classify abnormal sqlcode, and display the proportion of abnormal sqlcode in real time.

[4] Rollback rate: the proportion of rolled back transactions in the total transactions, and timely find transactions with high rollback.

[5] TPS: transaction volume per second, which is an important indicator of database service capability.

[6] RSPT: the average response time per second of a database. It is an important indicator to measure the service capability of a database.

[7] Uncommitted transactions: the number of uncommitted transactions in the current connection for a long time. Quickly locate what kind of transactions are suspended, which affects the overall performance of the database.

[8] Top slow SQL: monitor the response time of top level statement execution, find slow statements and predict the response time trend.

(2)Resource monitoring

[1] Sqlcode type: quickly determine abnormal behaviors such as database and tablespace according to different sqlcode return code types.

[2] Affected rows: the number of rows affected by each SQL statement. It is an important indicator to evaluate the impact of SQL statements on the database.

[3] Channel monitoring: monitor whether all connections are normal.

[4] DDL statement monitoring: monitor all DDL statements.