Security Threats Detection Of The Social Blogosphere
Objective
Our customer wanted to develop a security service with the ability to to analyze third-party website content such as forums or blogs in order to detect planned hacker attacks and similar illegal actions. As a result of Axmor’s extensive experience in Big Data processing, Social Network Mining and User Opinion Analysis, (all of paramount importance to our client) Axmor developers were approached to design the security service prototype.
Solution
Axmor designed and developed a security service prototype which provides data mining from heterogeneous Internet resources, and sends out user alerts and notifications regarding threats, planned hacker attacks and other nefarious activities. The security service contains the following modules: web interface for data collection parameters management and analysis, third-party website web crawler servers, database repository and potential threat detection notification system. This service is designed for banks and companies to monitor and detect potential threats that can lead to significant damage to their business.
Service administrators register and set up new user profiles according to the selected tariff plan.
In the personal account, users manage a list of websites for to scan and configure content, filtering via the use of keywords to obtain the desired data. A keyword could be, for example, the name of the bank, a bank card number, a person’s name, and many more options. Users also specify e-mail addresses in order to receive prompt notifications of potential threats.
The developed web crawling service scans third-party websites by means of emulating user behavior to bypass real-time bot resource protection. In addition, to provide traffic anonymity, we use an onion routing distributed network (Tor).
Data collected from scanned websites are stored in encrypted form in the database repository. We have developed a query builder that creates and saves complex search queries using a given list of keywords and logical operators. Created queries are used to search the scanned data stored in the database. At the same time, the developed solution uses morphology based built in SQL-like search engine. All duplicated data is deleted from the dataset, leaving only the unique data.
Based on the scanned data, users receive e-mail alerts if the developed security service has detected data related to specific keywords.
Users can review data for a selected time period pertaining to the number and content of sent alerts.
Axmor integrated the developed security service with Twitter, allowing users to publish received potential threat alerts on their Twitter account. This functionality is especially beneficial for banks, as they can instantly inform their Twitter followers about bank card numbers in the possession of scammers.
