Add part 2 : CSV parser

EXERCISE_fr.adoc

@@ -34,12 +34,102 @@ Afficher le contenu d'un fichier dans la console
 
 * Créer une nouvelle classe `fr.lernejo.file.Cat`
 * Créer une fonction `main` de telle sorte que :
-** Le programme prenne un chemin (relatif ou absolu) comme unique argument
+** Le programme prent un chemin (relatif ou absolu) comme unique argument
 ** En cas d'absence d'argument, le programme sort avec le code 3 et le message "Missing argument" dans la sortie standard
 ** Dans le cas de deux arguments ou plus, le programme sort avec le code 4 et le message "Too many arguments" dans la sortie standard
 ** Dans le cas où le fichier indiqué n'existe pas, le programme sort avec le code 5 et le message "File not found"
 ** Dans le cas où le chemin indiqué est un répertoire, le programme sort avec le code 6 et le message "A file is required"
 ** Dans le cas où le fichier fait plus de 3KiB (soit 3*1024 = 3072 bytes), le programme sort avec le code 7 et le message "File too large"
 ** Dans le cas où le fichier est lisible et fait moins de 3KiB, afficher le contenu du fichier dans la console et sortir
 
-* ✔️ Running `java -jar target/maven-training.jar README.md` should display the content of the `README.md` file
+* ✔️ Running `java -cp target/maven-training.jar fr.lernejo.file.Cat README.md` should display the content of the `README.md` file
+
+== Partie 2
+
+Lire un fichier de données au format CSV, et réaliser un calcul simple.
+
+* Créer une nouvelle classe `fr.lernejo.file.CsvReader`
+* Créer une fonction `main` de telle sorte que :
+** Le programme prent ces paramètres :
+*** Un chemin vers un fichier (CSV)
+*** Une date de début au format `YYYY-MM-dd` (inclue)
+*** Une date de fin (exclue)
+*** Un nom de métrique parmi
+**** `temperature_2m` pour température (2ᵉ colonne)
+**** `pressure_msl` pour la pression (4ᵉ colonne)
+**** `wind_speed_10m` pour la vitesse du vent (6ᵉ colonne)
+**** `direct_normal_irradiance_instant` pour l'irradiance (9ᵉ colonne)
+*** Un sélecteur : `NIGHT` ou `DAY`
+*** Un type d'agrégation : `SUM`, `AVG`, `MIN`, `MAX`
+** Le programme affiche la valeur calculée suivie de son unité et sort
+
+Le fichier CSV en entrée sera toujours structuré de la même manière à savoir :
+
+* 3 lignes de préambule
+* 1 ligne d'entêtes (avec nom des métriques et unités)
+* N lignes de données
+* Les colonnes seront toujours les mêmes, et dans le même ordre.
+
+Exemple de fichier CSV :
+
+[source,csv]
+----
+latitude,longitude,elevation,utc_offset_seconds,timezone,timezone_abbreviation
+52.54833,13.407822,38.0,0,GMT,GMT
+
+time,temperature_2m (°C),rain (mm),pressure_msl (hPa),cloud_cover (%),wind_speed_10m (km/h),soil_temperature_0_to_7cm (°C),is_day (),direct_normal_irradiance_instant (W/m²)
+2010-01-01T00:00,-2.6,0.00,996.9,100,16.0,-0.6,0,0.0
+2010-01-01T01:00,-2.7,0.00,996.4,100,16.3,-0.6,0,0.0
+2010-01-01T02:00,-2.7,0.00,996.2,100,16.3,-0.6,0,0.0
+2010-01-01T03:00,-2.7,0.00,996.1,100,15.4,-0.6,0,0.0
+2010-01-01T04:00,-2.7,0.00,996.0,100,14.8,-0.6,0,0.0
+2010-01-01T05:00,-2.7,0.00,996.1,100,14.8,-0.6,0,0.0
+2010-01-01T06:00,-2.8,0.00,996.1,100,15.0,-0.6,0,0.0
+2010-01-01T07:00,-2.7,0.00,996.3,100,14.3,-0.6,0,0.0
+2010-01-01T08:00,-2.7,0.00,996.7,100,13.8,-0.6,1,0.0
+2010-01-01T09:00,-2.4,0.00,997.2,100,13.7,-0.6,1,8.5
+2010-01-01T10:00,-2.2,0.00,997.2,100,13.2,-0.6,1,20.4
+2010-01-01T11:00,-2.0,0.00,997.3,100,13.8,-0.6,1,50.0
+2010-01-01T12:00,-1.7,0.00,997.6,100,14.0,-0.6,1,44.7
+2010-01-01T13:00,-1.6,0.00,997.6,100,14.3,-0.6,1,27.9
+2010-01-01T14:00,-1.7,0.00,998.1,100,12.8,-0.6,1,33.9
+2010-01-01T15:00,-2.1,0.00,999.1,100,10.5,-0.6,1,0.0
+2010-01-01T16:00,-2.5,0.00,999.6,100,10.6,-0.6,0,0.0
+2010-01-01T17:00,-2.8,0.00,999.9,100,10.8,-0.6,0,0.0
+2010-01-01T18:00,-3.2,0.00,1000.4,100,10.3,-0.6,0,0.0
+2010-01-01T19:00,-2.9,0.00,1001.1,100,10.5,-0.6,0,0.0
+2010-01-01T20:00,-3.0,0.00,1001.8,100,10.0,-0.6,0,0.0
+2010-01-01T21:00,-2.9,0.00,1002.3,100,9.5,-0.6,0,0.0
+2010-01-01T22:00,-2.9,0.00,1002.9,100,11.0,-0.5,0,0.0
+2010-01-01T23:00,-2.9,0.00,1003.4,100,8.7,-0.5,0,0.0
+2010-01-02T00:00,-3.0,0.00,1004.0,100,8.3,-0.5,0,0.0
+2010-01-02T01:00,-3.2,0.00,1004.5,100,9.1,-0.5,0,0.0
+2010-01-02T02:00,-3.4,0.00,1005.2,100,10.1,-0.5,0,0.0
+2010-01-02T03:00,-3.5,0.00,1005.6,93,11.2,-0.5,0,0.0
+2010-01-02T04:00,-3.7,0.00,1006.2,94,11.2,-0.5,0,0.0
+2010-01-02T05:00,-3.8,0.00,1007.2,99,10.9,-0.5,0,0.0
+2010-01-02T06:00,-3.8,0.00,1008.0,97,10.0,-0.5,0,0.0
+2010-01-02T07:00,-3.6,0.00,1008.9,72,10.0,-0.5,0,0.0
+2010-01-02T08:00,-3.6,0.00,1009.8,62,10.5,-0.5,1,0.0
+2010-01-02T09:00,-3.2,0.00,1010.6,64,11.0,-0.5,1,16.9
+2010-01-02T10:00,-2.7,0.00,1011.4,92,11.6,-0.5,1,25.4
+2010-01-02T11:00,-1.9,0.00,1012.0,100,10.9,-0.5,1,82.9
+2010-01-02T12:00,-1.4,0.00,1012.7,100,11.2,-0.5,1,84.7
+2010-01-02T13:00,-1.2,0.00,1013.5,100,14.3,-0.5,1,69.3
+2010-01-02T14:00,-1.4,0.00,1014.1,100,15.8,-0.5,1,73.5
+2010-01-02T15:00,-1.9,0.00,1015.2,93,15.0,-0.5,1,0.0
+2010-01-02T16:00,-2.0,0.00,1016.0,100,15.3,-0.5,0,0.0
+2010-01-02T17:00,-2.4,0.00,1016.6,89,13.9,-0.5,0,0.0
+2010-01-02T18:00,-3.2,0.00,1017.4,88,12.4,-0.5,0,0.0
+2010-01-02T19:00,-4.4,0.00,1018.0,97,11.2,-0.5,0,0.0
+2010-01-02T20:00,-5.1,0.00,1018.6,85,10.8,-0.5,0,0.0
+2010-01-02T21:00,-5.7,0.00,1018.8,92,10.9,-0.5,0,0.0
+2010-01-02T22:00,-5.7,0.00,1019.1,100,9.2,-0.5,0,0.0
+2010-01-02T23:00,-5.6,0.00,1019.4,100,7.3,-0.5,0,0.0
+----
+
+Un tel fichier est disponible peut être construit sur le site https://open-meteo.com.
+Le fichier ci-dessus a été téléchargé à l'URL https://archive-api.open-meteo.com/v1/archive?latitude=52.52&longitude=13.41&start_date=2010-01-01&end_date=2010-01-02&hourly=temperature_2m,rain,pressure_msl,cloud_cover,wind_speed_10m,soil_temperature_0_to_7cm,is_day,direct_normal_irradiance_instant&format=csv
+Customisation possible avec https://open-meteo.com/en/docs/historical-weather-api#start_date=2010-01-01&end_date=2010-01-02&hourly=temperature_2m,rain,pressure_msl,cloud_cover,wind_speed_10m,soil_temperature_0_to_7cm,is_day,direct_normal_irradiance_instant
+
+10 ans de données =~ 5 MiB

pom.xml

@@ -15,6 +15,7 @@
 
         <korekto-toolkit.version>1.4.2</korekto-toolkit.version>
         <logback.version>1.5.6</logback.version>
+        <juniversalchardet.version>2.5.0</juniversalchardet.version>
         <junit.version>5.10.3</junit.version>
         <assertj.version>3.26.3</assertj.version>
 
@@ -58,6 +59,11 @@
             <artifactId>logback-classic</artifactId>
             <version>${logback.version}</version>
         </dependency>
+        <dependency>
+            <groupId>com.github.albfernandez</groupId>
+            <artifactId>juniversalchardet</artifactId>
+            <version>${juniversalchardet.version}</version>
+        </dependency>
 
         <dependency>
             <groupId>org.junit.jupiter</groupId>

src/main/java/com/github/lernejo/korekto/grader/load_file/LaunchingContext.java

@@ -2,16 +2,11 @@
 
 import com.github.lernejo.korekto.toolkit.GradingConfiguration;
 import com.github.lernejo.korekto.toolkit.GradingContext;
-import com.github.lernejo.korekto.toolkit.misc.OS;
-import com.github.lernejo.korekto.toolkit.misc.Processes;
 import com.github.lernejo.korekto.toolkit.partgrader.MavenContext;
 
-import java.io.BufferedInputStream;
 import java.io.IOException;
-import java.io.InputStream;
 import java.io.UncheckedIOException;
 import java.net.URISyntaxException;
-import java.nio.charset.StandardCharsets;
 import java.nio.file.Files;
 import java.nio.file.Path;
 import java.nio.file.Paths;
@@ -21,10 +16,14 @@
 import java.util.stream.IntStream;
 import java.util.stream.Stream;
 
+import static com.github.lernejo.korekto.grader.load_file.PathUtils.resourceToPath;
+
 public class LaunchingContext extends GradingContext implements MavenContext {
     private final List<String> dictionary;
+    private final List<WeatherComputationData> dataset = WeatherComputationData.Loader.load();
     private boolean compilationFailed;
     private boolean testFailed;
+    public static final Path DATA_FILE_PATH = resourceToPath("open-meteo-52.55N13.41E38m.csv").toAbsolutePath();
 
     public LaunchingContext(GradingConfiguration configuration) {
         super(configuration);
@@ -68,56 +67,6 @@ public Optional<Path> jarPath() {
         }
     }
 
-    public Processes.ProcessResult launchJava(Path workingDirectory, Path jarPath, String mainClass, String... arguments) {
-        Path binPath = Paths.get(System.getProperty("java.home")).resolve("bin");
-        final Path javaPath;
-        if (OS.WINDOWS.isCurrentOs()) {
-            javaPath = binPath.resolve("java.exe");
-        } else {
-            javaPath = binPath.resolve("java");
-        }
-        List<String> commandPrefix = List.of(
-            javaPath.toString(),
-            "-Duser.country=UK",
-            "-Duser.language=en",
-            "-cp",
-            jarPath.toString(),
-            mainClass
-        );
-        List<String> command = Stream.concat(commandPrefix.stream(), Arrays.stream(arguments)).toList();
-        ProcessBuilder processBuilder = new ProcessBuilder(command);
-        processBuilder.directory(workingDirectory.toFile());
-
-        try {
-            var process = processBuilder.start();
-            int exitCode = process.waitFor();
-            if (exitCode != 0) {
-                var stdout = readStream(process.getInputStream());
-                var stderr = readStream(process.getErrorStream());
-                return Processes.ProcessResult.Companion.error(exitCode, stderr + stdout);
-            }
-            var stdout = readStream(process.getInputStream());
-            return Processes.ProcessResult.Companion.success(stdout);
-        } catch (IOException | InterruptedException e) {
-            return Processes.ProcessResult.Companion.error(e);
-        }
-    }
-
-    public String readStream(InputStream inputStream) {
-        try (BufferedInputStream bis = new BufferedInputStream(inputStream)) {
-            bis.mark(1);
-            var firstByte = bis.read();
-            if (firstByte != -1) {
-                bis.reset();
-                return new Scanner(bis, StandardCharsets.UTF_8).useDelimiter("\\A").next();
-            } else {
-                return "";
-            }
-        } catch (IOException e) {
-            throw new UncheckedIOException(e);
-        }
-    }
-
     public String getRandomWord() {
         int index = getRandomSource().nextInt(dictionary.size());
         return dictionary.get(index);
@@ -140,4 +89,13 @@ public String writeRandomWords(Path path, int nbWords) {
             throw new UncheckedIOException(e);
         }
     }
+
+    public Set<WeatherComputationData> getDataset(int size) {
+        Set<WeatherComputationData> r = new HashSet<>();
+        while (r.size() != size) {
+            int index = getRandomSource().nextInt(dataset.size());
+            r.add(dataset.get(index));
+        }
+        return r;
+    }
 }

src/main/java/com/github/lernejo/korekto/grader/load_file/LoadFileGrader.java

@@ -1,6 +1,7 @@
 package com.github.lernejo.korekto.grader.load_file;
 
 import com.github.lernejo.korekto.grader.load_file.parts.Part1Grader;
+import com.github.lernejo.korekto.grader.load_file.parts.Part2Grader;
 import com.github.lernejo.korekto.toolkit.GradePart;
 import com.github.lernejo.korekto.toolkit.Grader;
 import com.github.lernejo.korekto.toolkit.GradingConfiguration;
@@ -59,12 +60,13 @@
                 "Compilation & Tests",
                 1.0D),
             new JacocoCoveragePartGrader<>("Code Coverage", 4.0D, 0.85D),
-            new Part1Grader("Part 1 - Cat program", 4.0D)
+            new Part1Grader("Part 1 - Cat program", 4.0D),
+            new Part2Grader("Part 2 - CSV reader", 6.0D)
         );
     }
 
     @Override
     public boolean needsWorkspaceReset() {
-        return false;
+        return true;
     }
 }

src/main/java/com/github/lernejo/korekto/grader/load_file/PathUtils.java

@@ -0,0 +1,16 @@
+package com.github.lernejo.korekto.grader.load_file;
+
+import java.net.URISyntaxException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+
+public class PathUtils {
+
+    public static Path resourceToPath(String resourceName) {
+        try {
+            return Paths.get(WeatherComputationData.class.getClassLoader().getResource(resourceName).toURI());
+        } catch (URISyntaxException e) {
+            throw new RuntimeException(e);
+        }
+    }
+}

src/main/java/com/github/lernejo/korekto/grader/load_file/StringUtils.java

@@ -0,0 +1,26 @@
+package com.github.lernejo.korekto.grader.load_file;
+
+import java.util.regex.Matcher;
+
+public class StringUtils {
+
+    public static String safeEscapeElide(String s) {
+        if (s == null) {
+            return "";
+        } else {
+            String s1 = s.trim().replaceAll("\n", Matcher.quoteReplacement("\\n"));
+            if (s1.length() > 80) {
+                return s1.substring(0, 77) + "...";
+            }
+            return s1;
+        }
+    }
+
+    public static String safeLowerTrim(String s) {
+        if (s == null) {
+            return "";
+        } else {
+            return s.trim().toLowerCase();
+        }
+    }
+}

src/main/java/com/github/lernejo/korekto/grader/load_file/WeatherComputationData.java

@@ -0,0 +1,25 @@
+package com.github.lernejo.korekto.grader.load_file;
+
+import com.github.lernejo.korekto.grader.load_file.vault.Vault;
+
+import java.util.List;
+
+import static com.github.lernejo.korekto.grader.load_file.PathUtils.resourceToPath;
+
+public record WeatherComputationData(String start, String end, String metric, String selector, String agg,
+                                     String result) {
+
+    public String asDescription() {
+        String period = Vault.isClear() ? start + '/' + end : "(period undisclosed)";
+        return period + '/' + metric + '/' + selector + '/' + agg;
+    }
+
+    public static final class Loader {
+        public static final Vault<List<WeatherComputationData>> vault = new Vault<>() {
+        };
+
+        public static List<WeatherComputationData> load() {
+            return vault.load(resourceToPath("vault/results.encrypted"), resourceToPath("vault/results.clear"));
+        }
+    }
+}