finalized example

Author: pmbittner

README.md

@@ -16,14 +16,16 @@ In the following we give a step by step example in how the library can be used t
   - sample variants randomly, or use a predefined set of variants for simulation,
   - generate variants for each step in the evolution history,
   - obtain the ground truth of generated variants.
-The examples source code can also be found in [GenerationExample.java](src/main/java/vevos/examples/GenerationExample.java).
-We also give a brief introduction of the key features of the library we use in the following example.
+
+The example's source code can also be found in [GenerationExample.java](src/main/java/vevos/examples/GenerationExample.java).
+A similar and executable version of this example can be found in [VEVOSBenchmark.java](src/main/java/vevos/examples/VEVOSBenchmark.java), which gathers some rudimentary data on runtime performance for Linux and Busybox variant simulation. 
 
 At the very begin of your program, you have to initialize the library:
 ```java
 VEVOS.Initialize();
 ```
 This initializes the libraries logging and binding to FeatureIDE.
+You may also set a log level for the library here via `Logger::setLogLevel`.
 
 We can then start by specifying the necessary paths to (1) the git repository of the input software product line, (2) the directory of the extracted ground truth dataset, (3) and a directory to which we want to generate variants. (We use case sensitive paths to also allow the generation of Linux variants under Windows).
 ```java
@@ -47,7 +49,7 @@ From the loaded `dataset`, we can obtain the available evolution step.
 An evolution step describes a commit-sized change to the input software product line, and is defined by the (child) commit performing a change to a previous (parent) commit.
 Note that the evolution steps are not ordered because commits in the input product-line repository might not have been ordered as the commits might have been extracted from different branches.
 Alternatively, we can also request a continuous history of evolution steps instead of an unordered set.
-Therefore, a `SequenceExtractor` is used to determine how the successfully extracted commits should ordered.
+Therefore, a `SequenceExtractor` is used to determine how the successfully extracted commits should be ordered.
 In this example, we use the `LongestNonOverlappingSequences` extractor to sort the commits into one single continuous history.
 Nevertheless, merge commits and error commits (where VEVOS/Extraction failed) are excluded from the history and thus, the returned list of commits has gaps.
 Because of these gaps, we obtain a list of sub-histories, where each sub-history is continuous but sub-histories are divided by merge and error commits.
@@ -63,6 +65,16 @@ final VariabilityHistory history = dataset.getVariabilityHistory(new LongestNonO
 /// Thus, we divide the history into sub-histories at each failed commit.
 final NonEmptyList<NonEmptyList<SPLCommit>> sequencesInHistory = history.commitSequences();
 ```
+A sequence extraction might fail, for example if all commits are unrelated and no continuous history could be derived.
+In case this happens, as an alternative, you can either iterate over all `evolutionSteps` or iterate over all success commits in isolation:
+```java
+final List<SPLCommit> successCommits = dataset.getSuccessCommits();
+```
+In particular, the `VariabilityDataset` provides:
+- _success commits_ for which the extraction of feature mappings and feature model succeeded,
+- _partial success_ commits for which part of the extraction failed; Usually, a partial success commit has feature mappings but no file presence condition and no feature model,
+- _error commits_ for which the extraction failed.
+
 To generate variants, we have to specify which variants should be generated.
 Therefore, a `Sampler` is used that returns the set of variants to use for a certain feature model.
 Apart from the possibility of introducing custom samplers, VEVOS/Simulation comes with two built-in ways for sampling:
@@ -91,6 +103,16 @@ final Sample variantsToGenerate = new Sample(List.of(
 
 Sampler variantsSampler = new ConstSampler(variantsToGenerate);
 ```
+For the generation of variants we have to be able to access the repository of the input software product line to retrieve its source code.
+We reference the repository with an instance of `SPLRepository`:
+```java
+/// in general:
+final SPLRepository splRepository = new SPLRepository(splRepositoryPath.path());
+/// for Busybox:
+final SPLRepository splRepository = new BusyboxRepository(splRepositoryPath.path());
+```
+Note that Busybox has a special subclass called `BusyboxRepository` that performs some necessary pre- and postprocessing on the product lines source code.
+
 We are now ready to traverse the evolution history to generate variants:
 ```java
 for (final NonEmptyList<SPLCommit> subhistory : history.commitSequences()) {
@@ -108,6 +130,7 @@ A `Lazy` caches its loaded value so loading is only performed once.
 (Loaded data that is not required anymore can and should be freed by invoking `Lazy::forget`.)
 As the extraction of feature model or presence condition might have failed, both types are again wrapped in an `Optional` that contains a value if extraction was successful.
 Let's assume the extraction succeeded by just invoking `orElseThrow` here.
+(However, if also partial success commits are considered, one might need a more careful procedure here.)
 ```java
         final Artefact pcs = loadPresenceConditions.run().orElseThrow();
         final IFeatureModel featureModel = loadFeatureModel.run().orElseThrow();
@@ -126,6 +149,17 @@ Here, we just instruct the generation to exit in case an error happens but we co
         final ArtefactFilter<SourceCodeFile> artefactFilter = ArtefactFilter.KeepAll();
         final VariantGenerationOptions generationOptions = VariantGenerationOptions.ExitOnError(artefactFilter);
 ```
+To generate variants, we have to access the source code of the input software product line, at the currently inspected commit.
+We thus checkout the current commit in the product line's repository:
+```java
+        try {
+            splRepository.checkoutCommit(splCommit);
+        } catch (final GitAPIException | IOException e) {
+            Logger.error("Failed to checkout commit " + splCommit.id() + " of " + splRepository.getPath() + "!", e);
+            return;
+        }
+```
+
 Finally, we may indeed generate our variants:
 ```java
         for (final Variant variant : variants) {
@@ -152,7 +186,18 @@ In contrast, the suffix is `.spl.csv` for ground truth presence conditions of th
 
                 /// We can also export the ground truth PCs of the variant.
                 Resources.Instance().write(Artefact.class, presenceConditionsOfVariant, variantDir.resolve("pcs.variant.csv").path());
-            } 
+            }
+        }
+```
+In case we use Busybox as our input product line, we have to clean its repository as a last step:
+```java
+        if (splRepository instanceof BusyboxRepository b) {
+            try {
+                b.postprocess();
+            } catch (final GitAPIException | IOException e) {
+                Logger.error("Busybox postprocessing failed, please clean up manually (e.g., git stash, git stash drop) at " + splRepository.getPath(), e);
+            }
+        }
 ```
 This was round-trip about the major features of VEVOS/Simulation. Further features and convencience methods can be found in our documentation.
 

src/main/java/vevos/examples/GenerationExample.java

@@ -1,6 +1,7 @@
 package vevos.examples;
 
 import de.ovgu.featureide.fm.core.base.IFeatureModel;
+import org.eclipse.jgit.api.errors.GitAPIException;
 import vevos.VEVOS;
 import vevos.feature.Variant;
 import vevos.feature.config.SimpleConfiguration;
@@ -12,6 +13,8 @@
 import vevos.functjonal.Result;
 import vevos.functjonal.list.NonEmptyList;
 import vevos.io.Resources;
+import vevos.repository.BusyboxRepository;
+import vevos.repository.SPLRepository;
 import vevos.util.Logger;
 import vevos.util.io.CaseSensitivePath;
 import vevos.variability.EvolutionStep;
@@ -26,6 +29,7 @@
 import vevos.variability.pc.options.VariantGenerationOptions;
 import vevos.variability.sequenceextraction.LongestNonOverlappingSequences;
 
+import java.io.IOException;
 import java.util.List;
 import java.util.Map;
 import java.util.Optional;
@@ -65,6 +69,10 @@ public static void main(final String[] args) throws Resources.ResourceIOExceptio
         }
         Logger.info("");
 
+        /// In case the sequence extraction fails, you might either use the evolutionSteps directly or
+        /// iterate over all commits in isolation:
+//        final List<SPLCommit> successCommits = dataset.getSuccessCommits();
+
         /// Now lets use the variant generator to generate variants.
         /// First, create a sampler that determines which variants to generate at each evolution step.
         Sampler variantsSampler;
@@ -88,6 +96,15 @@ public static void main(final String[] args) throws Resources.ResourceIOExceptio
             variantsSampler = new ConstSampler(variantsToGenerate);
         }
 
+        /// Access the input software-product line's repository.
+        /// In case you use Busybox as input SPL, use the BusyboxRepository class.
+        /// Busybox requires special pre- and postprocessing steps upon commit checkout, that are handled by the
+        /// BusyboxRepository class.
+        final SPLRepository splRepository =
+                new SPLRepository(splRepositoryPath.path());
+//                new BusyboxRepository(splRepositoryPath.path());
+
+
         /// For the entire history
         for (final NonEmptyList<SPLCommit> subhistory : history.commitSequences()) {
             for (final SPLCommit splCommit : subhistory) {
@@ -117,6 +134,14 @@ public static void main(final String[] args) throws Resources.ResourceIOExceptio
                 final ArtefactFilter<SourceCodeFile> artefactFilter = ArtefactFilter.KeepAll();
                 final VariantGenerationOptions generationOptions = VariantGenerationOptions.ExitOnError(artefactFilter);
 
+                /// Checkout the considered commit of the input SPL to access its source code.
+                try {
+                    splRepository.checkoutCommit(splCommit);
+                } catch (final GitAPIException | IOException e) {
+                    Logger.error("Failed to checkout commit " + splCommit.id() + " of " + splRepository.getPath() + "!", e);
+                    return;
+                }
+
                 for (final Variant variant : variants) {
                     /// Let's put the variant into our target directory but indexed by commit hash and its name.
                     final CaseSensitivePath variantDir = variantsGenerationDir.resolve(splCommit.id(), variant.getName());
@@ -143,6 +168,14 @@ public static void main(final String[] args) throws Resources.ResourceIOExceptio
                                 result.getFailure());
                     }
                 }
+
+                if (splRepository instanceof BusyboxRepository b) {
+                    try {
+                        b.postprocess();
+                    } catch (final GitAPIException | IOException e) {
+                        Logger.error("Busybox postprocessing failed, please clean up manually (e.g., git stash, git stash drop) at " + splRepository.getPath(), e);
+                    }
+                }
             }
         }
     }

src/test/java/vevos/VEVOSBenchmark.java …/java/vevos/examples/VEVOSBenchmark.javasrc/test/java/vevos/VEVOSBenchmark.java renamed to src/main/java/vevos/examples/VEVOSBenchmark.java src/test/java/vevos/VEVOSBenchmark.java renamed to src/main/java/vevos/examples/VEVOSBenchmark.java

@@ -1,7 +1,7 @@
-package vevos;
+package vevos.examples;
 
 import de.ovgu.featureide.fm.core.base.IFeatureModel;
-import org.junit.Test;
+import vevos.VEVOS;
 import vevos.feature.Variant;
 import vevos.feature.sampling.FeatureIDESampler;
 import vevos.feature.sampling.Sample;
@@ -37,27 +37,25 @@ private record Repo(
             CaseSensitivePath groundTruthDatasetPath,
             CaseSensitivePath variantsGenerationDir,
             Function<CaseSensitivePath, SPLRepository> createRepo,
-            Consumer<SPLRepository> prepare,
             Consumer<SPLRepository> cleanup) {}
 
-    private static final CaseSensitivePath WIN_SUBMISSION_DIR = CaseSensitivePath.of("/mnt/c/Users/Paul Bittner/Documents/MyDocuments/Projects/VEVOS/Submission/Extraction/extraction-results/");
-    private static final CaseSensitivePath WSL_VARIANT_EVOLUTION_DATASETS_DIR = CaseSensitivePath.of("../variantevolution_datasets/");
+    /// TODO: Specify your paths here.
+    private static final CaseSensitivePath SPL_REPOS_DIR = CaseSensitivePath.of("path/to/SPL/repos");
+    private static final CaseSensitivePath DATASETS_DIR = CaseSensitivePath.of("path/to/datasets/");
+    private static final CaseSensitivePath VARIANT_GENERATION_DIR = CaseSensitivePath.of("path/to/datasets/");
 
     private static final Repo LINUX = new Repo(
-            WSL_VARIANT_EVOLUTION_DATASETS_DIR.resolve("linux"),
-            WIN_SUBMISSION_DIR.resolve("linux"),
-            CaseSensitivePath.of("../dockerbrezel/linux"),
+            SPL_REPOS_DIR.resolve("linux"),
+            DATASETS_DIR.resolve("linux"),
+            VARIANT_GENERATION_DIR.resolve("linux"),
             path -> new SPLRepository(path.path()),
-            s -> {},
             s -> {}
     );
     private static final Repo BUSYBOX = new Repo(
-            WSL_VARIANT_EVOLUTION_DATASETS_DIR.resolve("busybox/busybox"),
-            WSL_VARIANT_EVOLUTION_DATASETS_DIR.resolve("busybox/variability"),
-//            WIN_SUBMISSION_DIR.resolve("busybox"),
-            CaseSensitivePath.of("../dockerbrezel/busybox"),
+            SPL_REPOS_DIR.resolve("busybox"),
+            DATASETS_DIR.resolve("busybox"),
+            VARIANT_GENERATION_DIR.resolve("busybox"),
             path -> new BusyboxRepository(path.path()),
-            s -> { },
             s -> {
                 BusyboxRepository b = Cast.unchecked(s);
                 try {
@@ -76,7 +74,7 @@ private static String logTime(final String task, final double seconds) {
         return msg;
     }
 
-    private static void resultEntry(final StringBuilder builder, String entry) {
+    private static void resultEntry(final StringBuilder builder, final String entry) {
         builder.append(entry).append("\r\n");
     }
 
@@ -118,7 +116,6 @@ public static void benchmark(final Repo repo) throws Exception {
         for (final SPLCommit splCommit : subhistory) {
             Logger.info("-- Processing commit " + splCommit.id() + " --");
             splRepo.checkoutCommit(splCommit);
-            repo.prepare.accept(splRepo);
 
             clock.start();
             final Lazy<Optional<IFeatureModel>> loadFeatureModel = splCommit.featureModel();
@@ -193,13 +190,26 @@ public static void benchmark(final Repo repo) throws Exception {
         TextIO.write(repo.variantsGenerationDir.resolve("benchmarkdata.txt").path(), result);
     }
 
-    @Test
-    public void benchmarkLinux() throws Exception {
+    public static void benchmarkLinux() throws Exception {
         benchmark(LINUX);
     }
 
-    @Test
-    public void benchmarkBusybox() throws Exception {
+    public static void benchmarkBusybox() throws Exception {
         benchmark(BUSYBOX);
     }
+
+    public static void main(final String[] args) throws Exception {
+        if (args.length < 1) {
+            Logger.error("Expected exactly one argument that either is \"busybox\" or \"linux\"!");
+        }
+
+        final String repoName = args[0];
+        if ("linux".equalsIgnoreCase(repoName)) {
+            benchmarkLinux();
+        } else if ("busybox".equalsIgnoreCase(repoName)) {
+            benchmarkBusybox();
+        } else {
+            Logger.error("Unknown repository " + repoName + "! Expected \"busybox\" or \"linux\"!");
+        }
+    }
 }