/ Case introduction 2020In May, a large domestic joint-stock bank completed the migration of its core business system from IBM AS400 stand-alone to x86 Linux application service cluster.On May 3, the new core business system was put into operation. After three months of continuous and stable operation, it was officially announced that the new core business system was successfully put into operation in August. The new core business system takes the lead in adopting x86 + independent distributed database in China, and takes the lead in realizing a breakthrough among large and medium-sized banks in China. / Project background The scenario of the service system of a large joint-stock bank before migration is: on an AS400 production machine, both the core business (the business system developed by RPG) and the DB2 database are running, that is, the database and the business system are on the same machine.With the development of banking business system and the continuous expansion of transaction volume and business volume, it is difficult for a single machine to meet all business needs and data storage, and it is not easy to expand horizontally.Therefore, the bank is ready to convert its core business from RPG development to Java language development and run on the server based on X8 architecture.DB2 of the old AS400 database will be transformed into a multi node distributed database, which also runs on X86_On a 64 architecture server. / Customer pain points 1: How to realize business migration without downtime：Nowadays, the business system of large and medium-sized banks must provide all-weather continuous external service capability. How to ensure that the upgrading and iteration of the business system will not affect the actual use experience of users, but also make the business system continuously improve in the upgrading and transformation is the consideration of the comprehensive quality of a bank. Therefore, the migrated data and production data must be synchronized in real time, and services can be provided immediately after the migration is completed.
2.How can the migrated data be 100% correct：In the process of data migration, how to ensure that all user account data information can be migrated correctly, how to ensure that the converted data is 100% correct in the face of complex rule conversion scenarios, and how to minimize the impact on users.
3.How to divide databases and tables from stand-alone database to distributed cluster database：Before migration, all data is stored in the same database of a single machine, and after migration, all data is stored in dozens of X86 distributed clusters.How to migrate different data to the tables corresponding to different machines in the distributed x86 distributed database and ensure the integrity of the data is a great challenge.
4.How to solve the problems of large field data, heterogeneous table data and missing data fragmentation keys in business data migration： Because there are mixed types (O, P, etc.) in large fields, special transcoding and parsing operations are required.Heterogeneous table components include complex scenarios such as common heterogeneous (same table name, different structure), heterogeneous table name (different table names), one table splitting multiple tables, multiple tables in one, etc.Migrate core business database to x86_64 the distributed database needs to supplement the fragment key field for the migrated data, and some table data cannot generate fragment keys through their own data. It needs to be searched according to the mapping fields in the table. / Solution Firstly, by installing the log acquisition program on the backup machine, the collected log data is copied to the deployment environment of the tool in real time, the changed data of the database is restored through the analysis of the log, and then the operation statements of the target database are generated through various conversion and patch key operations, so as to realize the real-time synchronization between the production environment and the distributed database environment.At the same time, in the process of continuous incremental change of data, incremental verification of data will be carried out to ensure the consistency of final results. / Program advantages 1. Zero downtime       The whole migration process is from log collection, conversion processing to synchronization to distributed database. The data synchronization process has only a second delay.When the existing production database stops external services, the new distributed database access can directly provide external services, so as to ensure the uninterrupted business of the whole bank. 2. 100% accuracy       In the whole process of data synchronization, the correctness of the synchronization data of the target database is verified by continuously extracting the data loaded in the target database for incremental comparison. The possible errors are counted and summarized, and the primary data repair is carried out after manual confirmation after the final migration, so as to ensure the consistency of the final results and achieve 100% accuracy. 3. Safety and high reliability       When obtaining the migration tool log, you only need to access the log file of the backup database in the production system, which has no invasion to the production database and fully ensures the security of the production system.At the same time, the whole link of the tool supports breakpoint recovery to avoid abnormal events such as downtime and hardware failure during migration, so as to ensure the continuous and reliable migration.