SkillGraph – Self-Learning Technology
Editor’s note: This is an update from the SkillPages Engineering Team to the development of SkillGraph. See our previous posts for background information on SkillGraph.
Learning is a continuous human process and we apply the same principle to SkillGraph. As more skills and opportunities are added, and more people interact with these entries on the platform, the graph absorbs this into its knowledge base. The nodes in the graph and edges that connect them continuously evolve and grow. The rules that govern its structure and its use, as well as the semantic understanding behind them are in a constant flux.
We mix traditional knowledge based approaches with sophisticated “training” methods. User feedback is an important part of this process. If a member enters ambiguous wording for a skill, they are invited to choose their classification from a list of intelligent suggestions. For example we might get a skill added for a ‘Tester’ but we don’t know exactly what area to classify them in if we don’t know what they test. However, in this scenario the SkillGraph can evolve as more and more members, often sharing similar backgrounds, choose to categorize themselves as a ‘Software Test Engineer’. Going forward, the graph can make more confident assertions about subsequent users using the same piece of vocabulary to describe their skill.
Similarly user rejections are just as helpful for the SkillGraph to self-learn. Given rules are often adapted or even removed when a significant number of members are recorded as rejecting the categorizations associated to a particular rule in favour of others. Take for example the word ‘publication’, based on a set of rules around stemming (a programmatic process for reducing words to common base forms), our algorithm understood this to have the same meaning as the word ‘public’. However as members with skill data containing ‘publication’ continuously favoured results from those within the field of publishing over say the public sector or public relations, the SkillGraph learnt that ‘publication’ had little to do with ‘public’ and more to do with ‘publish’ and so adjusted its rules accordingly.
All user selections and rejections are recorded as this is valuable data to feed back to further inform and develop the SkillGraph. As a result, the graph is in a continual state of improvement. With the variety and volume of new skills growing every day, as well as the interactions around them, the SkillGraph becomes more dense and rich, resulting in more accurate and relevant classifications.