Dr. Andrew Adams, Deputy Director of the Centre for Business Information Ethics, Meiji University
Joint work with Toru Nakamura, Shinsaku Kiyomoto and Kiyoshi Murata
Topic: Effectiveness and Acceptability Evaluation for a Machine Learning-Based Privacy Setting Prediction Scheme
Abstract: Privacy settings for online systems are notoriously complicated for users. The number of settings is often very large, and in many systems these settings are distributed in different places around a user’s profile. As part of a larger project to provide users with greater control over their personal information, along the lines of Searls’ Vendor Relationship Management approach, the authors have investigated usability and user acceptability of a system of eighty settings permitting or limiting the sharing of information from a user’s system. Since this is a large number of settings, a machine learning algorithm was applied to a sample of users’ choices to create a prediction model matching users’ desired settings from a smaller number of answers. A support vector machine algorithm identified five questions from which the other 75 could be predicted with a reliability of over 90% from test data collected alongside the training data, using online participants.
This prediction system was then tested on three groups using an in-person experiment. Group A were presented with the five “prediction” questions, then shown the other 75 questions with the “prediction” shown and asked to accept or change the prediction. Group B were a control group given the eighty questions in the original order with no suggestions. Group C were give the eighty questions with the “prediction” questions first, but then asked to complete the other 75 without being shown their prediction.
All participants were also asked to evaluate the privacy settings system with regards to its utility to them, it understandability, and their level of desire to have such a system made available to them.
This talk presents the results of the experiment in terms of the accuracy of the prediction system and of any differences between accuracy between the experimental groups, as well as the outcomes of the follow-up questionnaire.