metaBup

 

metaBup validation

 
metaBup has been validated with real users in a research experiment. 11 users participated. This section contains the following: formulation of the expected DSL requirements, instructions of the experiment, and questionnaires. Questionnaire 1 was answered by all the participants, in order to convey their opinions on the first version of the generated modelling tool. Questionnaire 2 was only answered by those participants that opted for generating a second version of the modelling tool. Mandatory questions are marked with an asterisk.

DSL requirements

We want to model domestic networks in which a Service Provider (ISP) provides Internet access to its clients by means of local networks. Each client hosts a domestic net, connected to a unique local network via a modem. Each client can also have a router (which can be enabled for WiFi connections) with a number of ports to which different devices can be connected. The router has access to Internet through the modem.

Devices are connected to the net in a different way, depending on whether they are wireless or not. Printers, desktop computers, servers and fixed phones are connected through the router. They need to be plugged to one of the router ports, but the fixed phone which is directly connected to the router. On the other hand, wireles devices (laptops, Smart TVs and smartphones) can only access Internet through a WiFi network, which will be the one connected to the router directly.

Instructions

You can provide as many drawings as you deem necessary, each one illustrating domain examples according to your own criterion. They do not mandatorily have to be complete examples, but they can focus on specific aspects of the problem domain. For instance, you can make a diagram with just some local networks connected to Internet, another with the configuration of a WiFi network, etc.

The drawings do not need to include unnecessary elements for the aspect you want to model, while you should make as many drawings as necessary to illustrate all aspects of the domain. Each element in the palette should be used at least once in some drawing.

When making the drawings, please remind that the following properties are meaningful. (i) If you want to establish several kinds of connections between two objects, you should use different line styles. (ii) The relative position of objects is meaningful. This means that drawings can make use of containment, adjacency and overlapping of objects. (iii) You can add information to objects by attaching them labels with the following format: ≺ field-name ≻ = ≺ field-value ≻.

Questionnaire 1


Questionnaire 2

Exercises

Participant Fragments Time (m) Total objects Total references Total spatial relationships Distinct objects / fragment ME fragment edit time (s) Used spatial relationships Used connector styling Used labels Used connectors Demanded 2nd interaction Fragments Editor
aips 1 20 14,00 10,00 12,00 13,00 19 No No download download
fj 1 60 18,00 17,00 0,00 13,00 27 No No No No download download
plr 1 40 16,00 8,00 13,00 13,00 44
No No No download download
pga 2 30 19,00 22,00 0,00 8,50 49 No No No No download download
psjg 2 60 28,00 2,00 0,00 11,50 12 No No No No download download
sps 2 30 45,00 43,00 0,00 7,50 70 No No No No download download
vpl 2 30 19,00 14,00 11,00 7,50 78 No No No download download
dvm 3 37 22,00 18,00 2,00 5,67 66 No No No download download
jsc 3 120 25,00 14,00 11,00 6,67 128 No No download download
ams 4 40 39,00 23,00 26,00 9,00 77 No download download
jc 6 15 34,00 23,00 9,00 4,50 224 No No download download

Results

Consult the results of the evaluation (in Spanish).