Data Resource Management PlatformThe data resource management platform had many functions and could manage data resources in all aspects.
In a corporate setting, it could solve many pain points. For example, there were problems such as scattered data resources (data barriers between departments formed data islands), multi-source data (diverse technology platforms and storage technologies), inconsistent data standards, etc., which made it difficult to find and apply data. The data resource management platform could improve these situations.
Its product positioning was to face massive, multi-source, and isomerous data under a large number of users. It could check enterprise data resources, integrate and access various enterprise data resources, establish enterprise data resource catalog, provide a unified data management interface, and provide data sharing access interface for other users to manage enterprise data resources in a unified manner.
The value of the product included: first, it could solve the problem of enterprise data access and management, and deal with the data access and management of complex situations such as multi-source, heterogenious data/non-standardized interface; second, it could lower the technical threshold, and the data collection function could be realized through visual interface configuration; third, it could save enterprise costs, and could design storage solutions according to user data and business conditions, and support hierarchical and classified management of stored data.
The functions covered multiple modules:
- The external data source supports multiple types of data source adaptation, such as structured, semi-structured, and structured data types, including 20 + data sources such as Mysoul, Oracle, DB2, MogoDB, Hive, and so on.
- The purpose of data interrogation was to clarify the data to be integrated, the connection method, the IT environment, and other information to prepare for data integration. It also provided data interrogation templates to support the query and maintenance of data interrogation information.
- Data integration supports a variety of methods, such as data tables, API, EXCEL import, ETL, real-time data (Kafka), etc. There are full integration and lightweight integration modes to choose from. The integration process can extract, intercept, clean, and other processes of data as needed.
- The data storage supports the selection of multiple storage architecture based on data attributes and application requirements. It also supports data connection and configuration management of internal and external data sources.
- The data organization could manage the data by layers and categories, and support the creation and maintenance of data tables as well as the function of data labels.
- The data warehouse would display the data that had been classified and sorted in the form of a data catalog and support the query and viewing of data resources.
- The data service supports four kinds of data distribution services: data catalog service, API service, middle library service, and message distribution service.
In terms of technical architecture, the source side of the map was suitable for various data sources, and the target side supported a variety of storage methods. Through the platform, the closed-loop management of data interrogation, integration, storage, organization, digital warehouse catalog display, and distribution services was realized.
From the perspective of data flow, data sources of different types, format, and storage methods are collected to the platform through the data integration function; the original data collected in full or the lightly collected meta-data are stored and landed through appropriate storage methods; the data service shares the data in the form of data tables, middle-libraries, APIs, message distribution, etc.
In addition, there were other similar platforms such as CommVault's integrated data management platform, which could allow data management throughout the entire data life cycle, providing data protection, replication, archive, resource management, search, and other methods. Each functional module worked together to manage data with a single graphic interface, achieving seamless software integration and controlling data growth, costs, and risks. The integrated big data management platform launched by Global Software provides one-stop data management and service solutions for multiple parties, realizing the mutual recognition and sharing of data resources across regions, departments and levels. Its core functions include catalog management, supply and demand docking, resource management, data sharing, data opening, analysis and processing, etc.
"A Short History of the Future: Legends of the Intelligent Era" was equally exciting. Everyone was welcome to click and read it!