Skip to content

Commit 64fc32c

Browse files
Updated README and names
1 parent 25403c6 commit 64fc32c

20 files changed

Lines changed: 548 additions & 550 deletions

README.md

Lines changed: 56 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -1,46 +1,30 @@
1-
# SPL Variability Extraction
1+
# VEVOS: Ground Truth Extraction
22

33
<p>
4-
This tool offers functionality for extracting the feature model, presence conditions, and other metadata for a C-preprocessor-based SPL. The setup was configured and tested
5-
for BusyBox and Linux.
4+
This project offers functionality for extracting feature models, presence conditions, and other metadata for a kbuild-based software product line. The setup was configured and tested for BusyBox and Linux.
65
</p>
76

8-
9-
## Limitations
10-
There are some limitations to the variability extraction that should be mentioned.
11-
### SPL Versions
12-
Due to the plugins that are used by KernelHaven (i.e., KbuildMiner and KconfigReader), it is not possible to extract
13-
the variability of SPL revisions without prior setup of the operating system on which the extraction is run.
14-
More specifically, we were not able to get KernelHaven to run for linux versions of v5.0 or above. Older versions of linux (below v4), can also cause problems.
15-
This is due to changes in the build structure that require changes to KbuildMiner and KconfigReader. Similar problems probably also exist for Busybox, Coreboot, etc.
16-
`For this reason we offer scripts that setup a Docker container for extracting the data for BusyBox or Linux.`
17-
18-
### Operating System
19-
Due to the implementation of the VariabilityExtraction and KernelHaven, it is only possible to run the variability
20-
extraction on Linux (and possibly Mac). However, you can use the provided Docker setup, or your own virtual machine or
21-
Windows Subsystem for Linux, in order to run the extraction on any OS.
22-
237
## Quick Start using Docker
24-
In the following, we provide instructions on how to quickly extract the variability of Linux or Busybox with the provided
8+
In the following, we provide instructions on how to quickly extract the ground truth of Linux or Busybox with the provided
259
Docker setup.
2610

2711
### Requirements
2812
The only requirement is Docker. We provide batch and bash scripts that execute the necessary Docker setup and execution.
29-
We tested the Docker setup under Windows and Linux. We have not tested the setup on Mac, but you should be able to use
30-
follow the instructions for Linux.
13+
We tested the Docker setup under Windows and Linux.
14+
We have not tested the setup on Mac, but you should be able to use follow the instructions for Linux.
3115

3216
### Preparation
3317
#### Docker
3418
Docker must be installed on your system, and the Docker daemon must be running.
3519
For installation, follow the instructions given in the installation guide for your OS which you can find
3620
[here](https://docs.docker.com/get-docker/).
37-
Under Linux, you might additionally want to follow the optional
21+
Under Linux, you should follow the optional
3822
[post-installation instructions](https://docs.docker.com/engine/install/linux-postinstall/).
3923

4024
#### Repository
4125
Clone the repository to a location of your choice
4226
```
43-
git clone https://github.com/VariantSync/VariabilityExtraction.git
27+
git clone https://github.com/VariantSync/Extraction.git
4428
```
4529
Then, navigate to the repository's root directory in a terminal of your choice.
4630

@@ -50,9 +34,9 @@ Before the extraction can be executed, we have to build the Docker image. This c
5034
- Linux terminal: `./build-docker-image.sh`
5135
- Windows CMD: `build-docker-image.bat`
5236

53-
### Start the Variability Extraction in a Docker Container
54-
We provide bash and batch scripts that start the variability extraction and copy all data to
55-
_VariabilityExtraction/extraction-results_ once the extraction is complete, or has been stopped.
37+
### Start the Ground Truth Extraction in a Docker Container
38+
We provide bash and batch scripts that start the ground truth extraction and copy all data to
39+
_Extraction/extraction-results_ once the extraction is complete, or has been stopped.
5640
Start the extraction by executing the `start-extraction` script (see examples further below).
5741
The basic syntax is the following:
5842

@@ -90,26 +74,26 @@ Therefore, errors that appear in the log do not indicate a problem with the setu
9074

9175
#### Examples:
9276
```
93-
Extract the variability for all commits of BusyBox
77+
Extract the ground truth for all commits of BusyBox
9478
start-extraction.bat busybox
9579
./start-extraction.sh busybox
9680
97-
Extract the variability between two specific commits of Busybox
81+
Extract the ground truth between two specific commits of Busybox
9882
start-extraction.bat busybox b35eef5383a4e7a6fb60fcf3833654a0bb2245e0 7de0ab21d939a5a304157f75918d0318a95261a3
9983
./start-extraction.sh busybox b35eef5383a4e7a6fb60fcf3833654a0bb2245e0 7de0ab21d939a5a304157f75918d0318a95261a3
10084
101-
Extract the variability for the commit under revision tag v4.1 of Linux
85+
Extract the ground truth for the commit under revision tag v4.1 of Linux
10286
start-extraction.bat linux v4.1
10387
./start-extraction.sh linux v4.1
10488
105-
Extract the variability for all commits between two minor revisions of Linux
89+
Extract the ground truth for all commits between two minor revisions of Linux
10690
start-extraction.bat linux v4.3 v4.4
10791
./start-extraction.sh linux v4.3 v4.4
10892
```
10993

110-
### Stopping the Variability Extraction
111-
You can stop the Docker container in which the variability extraction is running at any time. In this case, all
112-
collected data will be copied to _VariabilityExtraction/extraction-results/_ as if the extraction finished successfully.
94+
### Stopping the Ground Truth Extraction
95+
You can stop the Docker container in which the ground truth extraction is running at any time. In this case, all
96+
collected data will be copied to _Extraction/extraction-results/_ as if the extraction finished successfully.
11397

11498
- Windows CMD:
11599
- `stop-extraction.bat busybox`
@@ -119,30 +103,44 @@ collected data will be copied to _VariabilityExtraction/extraction-results/_ as
119103
- `./stop-extraction.sh linux`
120104

121105
### Custom Configuration
122-
You can find the properties files used by Docker under VariabilityExtraction/docker-resources. By changing the properties
123-
for BusyBox or Linux respectively, you can adjust the variability extraction (e.g., change the log level,
106+
You can find the properties files used by Docker under Extraction/docker-resources. By changing the properties
107+
for BusyBox or Linux respectively, you can adjust the ground truth extraction (e.g., change the log level,
124108
number of threads, etc.). For your convenience, we set all properties to default values. __Note that you have to rebuild the Docker image in order for the changes to take effect__.
125109

126-
This can also be used in case you want to extract the variability for any other SPL besides Linux and BusyBox. However,
110+
This can also be used in case you want to extract the ground truth for any other SPL besides Linux and BusyBox. However,
127111
please note that this requires the correct configuration of KernelHaven and possibly other preprocessing steps, that
128112
are not included in this project. In addition, the extractors that are used internally, i.e., KbuildMiner and KconfigReader
129113
might not be applicable to the chosen SPL. In this case, custom extractors have to be implemented and added to the dependencies
130114
in the _pom.xml_ file.
131115

132116
### Clean-Up
133-
You clean up all created images, container, and volumes via `docker system prune -a`. __DISCLAIMER: This will remove ALL docker objects, even the ones not related to variability extraction__. If you have other images, containers, or volumes that you do not want to loose, you can run the docker commands that refer to the objects related to the variability extraction.
134-
- Image: `docker rmi variability-extraction`
117+
You clean up all created images, container, and volumes via `docker system prune -a`. __DISCLAIMER: This will remove ALL docker objects, even the ones not related to ground truth extraction__. If you have other images, containers, or volumes that you do not want to loose, you can run the docker commands that refer to the objects related to the ground truth extraction.
118+
- Image: `docker rmi extraction`
135119
- Container:
136-
- `docker container rm variability-extraction-busybox`
137-
- `docker container rm variability-extraction-linux`
120+
- `docker container rm extraction-busybox`
121+
- `docker container rm extraction-linux`
138122
- Volume:
139123
- `docker volume rm extraction-busybox`
140124
- `docker volume rm extraction-linux`
141125

142126
## Custom System Setup
143-
If you want to run the variability extraction without Docker, you will have to first set up the environment in which
127+
If you want to run the ground truth extraction without Docker, you will have to first set up the environment in which
144128
the extraction is executed.
145129

130+
### Limitations
131+
There are some limitations to the ground truth extraction that should be mentioned.
132+
#### SPL Versions
133+
Due to the plugins that are used by KernelHaven (i.e., KbuildMiner and KconfigReader), it is not possible to extract
134+
the ground truth of SPL revisions without prior setup of the operating system on which the extraction is run.
135+
More specifically, we were not able to get KernelHaven to run for linux versions of v5.0 or above. Older versions of linux (below v4), can also cause problems.
136+
This is due to changes in the build structure that require changes to KbuildMiner and KconfigReader. Similar problems probably also exist for Busybox, Coreboot, etc.
137+
`For this reason we offer scripts that setup a Docker container for extracting the data for BusyBox or Linux.`
138+
139+
#### Operating System
140+
Due to the implementation of the Ground Truth Extraction and KernelHaven, it is only possible to run the ground truth
141+
extraction on Linux (and possibly Mac). However, you can use the provided Docker setup, or your own virtual machine or
142+
Windows Subsystem for Linux, in order to run the extraction on any OS.
143+
146144
### Requirements
147145
- build-essential packages or similar (depending on OS)
148146
- git
@@ -264,33 +262,33 @@ Clone the repository of the SPL that you want to analyze (e.g., Linux):
264262
git clone https://github.com/torvalds/linux.git
265263
```
266264

267-
Clone the VariabilityExtraction repository:
265+
Clone the Extraction repository:
268266
```
269-
git clone [email protected]:mse/VariantSync/VariabilityExtraction.git
267+
git clone [email protected]:mse/VariantSync/Extraction.git
270268
```
271269

272270
Navigate into the repo:
273-
```cd VariabilityExtraction```
271+
```cd Extraction```
274272

275273
Build the jar:
276274
```mvn package```
277275

278276
Copy the required files to the work dir:
279277
```
280-
IF the VariabilityExtractionRepo is part of your work dir run:
278+
IF the ExtractionRepo is part of your work dir run:
281279
282-
cp target/VariabilityExtraction-*-jar-with* src/main/resources/KernelHaven.jar src/main/resources/variability_analysis_Linux.properties ..
280+
cp target/Extraction-*-jar-with* src/main/resources/KernelHaven.jar src/main/resources/extraction_linux.properties ..
283281
284282
ELSE
285283
286-
cp target/VariabilityExtraction-*-jar-with* src/main/resources/KernelHaven.jar src/main/resources/variability_analysis_Linux.properties WORKDIR/
284+
cp target/Extraction-*-jar-with* src/main/resources/KernelHaven.jar src/main/resources/extraction_linux.properties WORKDIR/
287285
```
288286

289287
Navigate back to the WORKDIR:
290288
```cd WORKDIR```
291289

292290
### Configuration
293-
Open the properties file `variability_analysis_Linux.properties` in an editor of your choice.
291+
Open the properties file `extraction_linux.properties` in an editor of your choice.
294292

295293
Adjust the path to the linux sources (absolute path): ```source_tree = WORKDIR/linux```
296294

@@ -299,7 +297,7 @@ Example: ```source_tree = /home/alice/linux-analysis/linux```
299297
Save the file.
300298

301299
### Validation
302-
The easiest way to check whether (almost) everything is set up correctly and whether it is possible to extract the variability for a specific Linux commit, is to run `make allyesconfig prepare` in the linux sources directory.
300+
The easiest way to check whether (almost) everything is set up correctly and whether it is possible to extract the ground truth for a specific Linux commit, is to run `make allyesconfig prepare` in the linux sources directory.
303301

304302
Navigate to the linux sources
305303
```cd linux```
@@ -310,25 +308,25 @@ Checkout the desired commit or revision, e.g.,
310308
Run make:
311309
```make allyesconfig prepare```
312310

313-
If no errors are thrown, the VariabilityExtraction *should* be successful for this commit. If you are able to complete the preparation for at least one commit, your system should be set up correctly.
311+
If no errors are thrown, the extraction *should* be successful for this commit. If you are able to complete the preparation for at least one commit, your system should be set up correctly.
314312

315313
Navigate back to working directory
316314
```cd ..```
317315

318-
### Variability Extraction
319-
You can run the variability extraction for a range of commits by specifying commit ids:
316+
### Ground Truth Extraction
317+
You can run the ground truth extraction for a range of commits by specifying commit ids:
320318
```
321-
java -jar VariabilityExtraction-1.0.0-jar-with-dependencies.jar variability_analysis_Linux.properties *START_COMMIT_ID* *END_COMMIT_ID*
319+
java -jar Extraction-1.0.0-jar-with-dependencies.jar extraction_linux.properties *START_COMMIT_ID* *END_COMMIT_ID*
322320
```
323321
or by specifying revision tags:
324322
```
325-
java -jar VariabilityExtraction-1.0.0-jar-with-dependencies.jar variability_analysis_Linux.properties *START_REVISION_TAG* *END_REVISION_TAG*
323+
java -jar Extraction-1.0.0-jar-with-dependencies.jar extraction_linux.properties *START_REVISION_TAG* *END_REVISION_TAG*
326324
```
327-
For example, to extract the variability for all commits between Linux v4.5 and Linux v4.6, the following is possible:
325+
For example, to extract the ground truth for all commits between Linux v4.5 and Linux v4.6, the following is possible:
328326
```
329-
java -jar VariabilityExtraction-1.0.0-jar-with-dependencies.jar variability_analysis_Linux.properties b562e44f507e863c6792946e4e1b1449fbbac85d 2dcd0af568b0cf583645c8a317dd12e344b1c72a
327+
java -jar Extraction-1.0.0-jar-with-dependencies.jar extraction_linux.properties b562e44f507e863c6792946e4e1b1449fbbac85d 2dcd0af568b0cf583645c8a317dd12e344b1c72a
330328
331-
java -jar VariabilityExtraction-1.0.0-jar-with-dependencies.jar variability_analysis_Linux.properties v4.5 v5.6
329+
java -jar Extraction-1.0.0-jar-with-dependencies.jar extraction_linux.properties v4.5 v5.6
332330
```
333331

334-
By default, the files with the extracted variability are stored under `WORKDIR/evolution-analysis/output/`
332+
By default, the files with the extracted ground truth are stored under `WORKDIR/evolution-analysis/output/`

build-docker-image.bat

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
docker build -t variability-extraction .
1+
docker build -t extraction .
22
@pause

build-docker-image.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
#! /bin/bash
2-
docker build -t variability-extraction .
2+
docker build -t extraction .
File renamed without changes.
File renamed without changes.

pom.xml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,16 +5,16 @@
55
<modelVersion>4.0.0</modelVersion>
66

77
<!-- Adjust group id for anonymous projects -->
8-
<groupId>de.variantsync.subjects.extraction</groupId>
8+
<groupId>de.variantsync.vevos</groupId>
99
<!-- Adjust the project name -->
10-
<artifactId>VariabilityExtraction</artifactId>
10+
<artifactId>Extraction</artifactId>
1111
<version>1.0.0</version>
1212

1313
<properties>
1414
<!-- Adjust your java version here -->
1515
<java.version>8</java.version>
1616
<!-- Adjust the path to the class that contains the main-method, that is to be executed, here -->
17-
<path.to.main>${project.groupId}.VariabilityExtraction</path.to.main>
17+
<path.to.main>${project.groupId}.Extraction</path.to.main>
1818
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
1919
<maven.compiler.source>${java.version}</maven.compiler.source>
2020
<maven.compiler.target>${java.version}</maven.compiler.target>

src/main/java/de/variantsync/subjects/extraction/AnalysisTask.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@
2323
import java.util.stream.Collectors;
2424
import java.util.stream.Stream;
2525

26-
import static de.variantsync.subjects.extraction.VariabilityExtraction.*;
26+
import static de.variantsync.subjects.extraction.Extraction.*;
2727
import static java.nio.file.StandardCopyOption.REPLACE_EXISTING;
2828

2929
public class AnalysisTask implements Runnable {

0 commit comments

Comments
 (0)