You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the following, we describe how to build the Docker image and run the experiments in Docker containers.
3
+
In the following, we describe how to replicate the validation from our paper (Section 5) step-by-step.
4
+
The instructions explain how to build the Docker image and run the validation in a Docker container.
4
5
5
-
### Install Docker (if required)
6
-
How to install Docker depends on your operating system.
6
+
### 1. Install Docker (if required)
7
+
How to install Docker depends on your operating system:
7
8
8
-
#### Windows or Mac
9
-
You can find download and installation instructions [here](https://www.docker.com/get-started).
10
-
11
-
#### Linux Distributions
12
-
How to install Docker on your system, depends on your distribution. However, the chances are high that Docker is part of your distributions package database.
9
+
-_Windows or Mac_: You can find download and installation instructions [here](https://www.docker.com/get-started).
10
+
-_Linux Distributions_: How to install Docker on your system, depends on your distribution. The chances are high that Docker is part of your distributions package database.
13
11
Docker's [documentation](https://docs.docker.com/engine/install/) contains instructions for common distributions.
Then, navigate to the root of your local clone of this repository:
35
+
```shell
36
+
cd DiffDetective
37
+
```
38
+
39
+
### 3. Build the Docker Container
40
+
To build the Docker container you can run the `build` script corresponding to your operating system:
32
41
```
33
42
# Windows:
34
43
.\build.bat
35
44
# Linux/Mac (bash):
36
45
./build.sh
37
46
```
38
47
39
-
## Verification & Expected Output
48
+
## 4. Verification & Replication
49
+
50
+
### Running the Replication or Verification
51
+
To execute the replication you can run the `execute` script corresponding to your operating system with `replication` as first argument.
52
+
53
+
#### Windows:
54
+
`.\execute.bat replication`
55
+
#### Linux/Mac (bash):
56
+
`./execute.sh replication`
57
+
58
+
> WARNING!
59
+
> The replication will at least require an hour and might require up to a day depending on your system.
60
+
> Therefore, we offer a short verification (5-10 minutes) which runs DiffDetective on only four of the datasets.
61
+
> You can run it by providing "verification" as argument instead of "replication" (i.e., `.\execute.bat verification`, `./execute.sh verification`).
62
+
> If you want to stop the execution, you can call the provided script for stopping the container in a separate terminal.
63
+
> When restarted, the execution will continue processing by restarting at the last unfinished repository.
64
+
> #### Windows:
65
+
> `.\stop-execution.bat`
66
+
> #### Linux/Mac (bash):
67
+
> `./stop-execution.sh`
40
68
41
-
### Running the Verification
42
-
To run the verification you can run the script corresponding to your OS with `verification` as first argument. The verification should take about 10-20 minutes depending on your hardware.
43
-
```
44
-
# Windows:
45
-
.\execute.bat verification
46
-
# Linux/Mac (bash):
47
-
./execute.sh verification
48
-
```
49
69
The results of the verification will be stored in the [results](results) directory.
50
70
51
71
### Expected Output of the Verification
52
-
The aggregated results of the verification can be found in the following files.
72
+
The aggregated results of the verification/replication can be found in the following files.
73
+
The example file content shown below should match your results when running the _verification_.
74
+
(Note that the links below only have a target _after_ running the replication or verification.)
53
75
54
-
- The [speed statistics](results/validation/speedstatistics.txt) contain information about the total runtime, median runtime, mean runtime, and more:
76
+
- The [speed statistics](results/validation/current/speedstatistics.txt) contain information about the total runtime, median runtime, mean runtime, and more:
55
77
```
56
-
#Commits: 14527
57
-
Total commit process time is: 12.427866666666667min
58
-
Fastest commit process time is: df4a1fa9c5cc5d54a9347a2bf4843cae87a942f1___xorg-server___0ms
59
-
Slowest commit process time is: 9838b7032ea9792bec21af424c53c07078636d21___xorg-server___14578ms
60
-
Median commit process time is: 6dc71f6b2c7ff49adb504426b4cd206e4745e1e3___xorg-server___19ms
61
-
Average commit process time is: 51.330075032697735ms
78
+
#Commits: 24701
79
+
Total commit process time is: 14.065916666666668min
80
+
Fastest commit process time is: d86e352859e797f6792d6013054435ae0538ef6d___xfig___0ms
81
+
Slowest commit process time is: 9838b7032ea9792bec21af424c53c07078636d21___xorg-server___7996ms
82
+
Median commit process time is: f77ffeb9b26f49ef66f77929848f2ac9486f1081___tcl___13ms
83
+
Average commit process time is: 34.166835350795516ms
62
84
```
63
-
- The [classification results](results/validation/ultimateresult.metadata.txt) contain information about how often each pattern was found, and more.
85
+
- The [classification results](results/validation/current/ultimateresult.metadata.txt) contain information about how often each pattern was matched, and more.
edgeformat: org.variantsync.diffdetective.mining.formats.DirectedEdgeLabelFormat with org.variantsync.diffdetective.mining.formats.ReleaseMiningDiffNodeFormat
(Note that the above links only have a target after running the verification.)
100
-
The processing times might deviate.
121
+
122
+
Moreover, the results comprise the (LaTeX) tables that are part of our paper and appendix.
123
+
The processing times might deviate because performance depends on your hardware.
124
+
125
+
### (Optional) Running DiffDetective on Custom Datasets
126
+
You can also run DiffDetective on other datasets by providing the path to the dataset file as first argument to the execution script:
127
+
128
+
#### Windows:
129
+
`.\execute.bat path\to\custom\dataset.md`
130
+
#### Linux/Mac (bash):
131
+
`./execute.sh path/to/custom/dataset.md`
132
+
133
+
The input file must have the same format as the other dataset files (i.e., repositories are listed in a Markdown table). You can find [dataset files](docs/datasets.md) in the [docs](docs) folder.
101
134
102
135
## Troubleshooting
103
136
104
137
### 'Got permission denied while trying to connect to the Docker daemon socket'
105
138
`Problem:` This is a common problem under Linux, if the user trying to execute Docker commands does not have the permissions to do so.
106
139
107
-
`Fix:` You can fix this problem by either following the [post-installation instructions](https://docs.docker.com/engine/install/linux-postinstall/), or by executing the scripts in the replication package with elevated permissions (i.e., `sudo`)
140
+
`Fix:` You can fix this problem by either following the [post-installation instructions](https://docs.docker.com/engine/install/linux-postinstall/), or by executing the scripts in the replication package with elevated permissions (i.e., `sudo`).
108
141
109
142
### 'Unable to find image 'replication-package:latest' locally'
110
143
`Problem:` The Docker container could not be found. This either means that the name of the container that was built does not fit the name of the container that is being executed (this only happens if you changed the provided scripts), or that the Docker container was not built yet.
111
144
112
145
`Fix:` Follow the instructions described above in the section `Build the Docker Container`.
113
146
114
-
### No results after verification, or 'cannot create directory '../results/difftrees': Permission denied'
147
+
### No results after verification, or 'cannot create directory '../results/validation/current': Permission denied'
115
148
`Problem:` This problem can occur due to how permissions are managed inside the Docker container. More specifically, it will appear, if Docker is executed with elevated permissions (i.e., `sudo`) and if there is no [results](results) directory because it was deleted manually. In this case, Docker will create the directory with elevated permissions, and the Docker user has no permissions to access the directory.
116
149
117
-
`Fix:` If there is a _results_ directory delete it with elevated permission (e.g., `sudo rm -r results`).
150
+
`Fix:` If there is a _results_ directory, delete it with elevated permission (e.g., `sudo rm -r results`).
118
151
Then, create a new _results_ directory without elevated permissions, or execute `git restore .` to restore the deleted directory.
0 commit comments