updates

web.py

@@ -272,9 +272,10 @@ def web_data():
         of 56,292 additional lines, resulting in the complete exclusion of 2,203 documents from a total of 13,560
         documents (16.25%). Accordingly, we choose to not use terminal punctuation as a signal to remove lines.
         """),
-        Img(
-            src="path/to/sample_terminal_punctuation_removed.png",
-            alt="Sample documents with lines that are removed by the rule of terminal punctuation",
+        view_data(
+            "data/sample_terminal_punc.json",
+            0,
+            "Sample documents with lines that are removed by the rule of terminal punctuation",
         ),
         H4('2.1 Word "Javascript"'),
         P("""
@@ -284,9 +285,10 @@ def web_data():
         propose to refine the strategy by adding one more keyword to the word "javascript" to avoid false positives.
         The additional keyword could be any one of “enable” / “disable” / “require” / “activate” / “browser”.
         """),
-        Img(
-            src="path/to/sample_javascript_removed_kept.png",
-            alt="Sample documents that are removed by original C4 javascript rule but are kept after our refinement",
+        view_data(
+            "data/sample_java.jsonl",
+            0,
+            "Sample documents that are removed by original C4 javascript rule but are kept after our refinement",
         ),
         H4("2.2 Other Rules from RefinedWeb"),
         P("""
@@ -296,9 +298,10 @@ def web_data():
         - The line matches the pattern “r'^\\d+\\s+likes$'”,
         - The line contains only one word.
         """),
-        Img(
-            src="path/to/sample_refinedweb_rules_removed.png",
-            alt="Sample documents with lines that are removed by the RefinedWeb rules",
+        view_data(
+            "data/sample_refinedweb_line.json",
+            0,
+            "Sample documents with lines that are removed by the RefinedWeb rules",
         ),
         H4("2.3 Toxic Lines"),
         P("""
@@ -308,15 +311,19 @@ def web_data():
         line is in the first 3 lines or in the last 3 lines) to remove toxic lines. Specifically, we do not only consider 
         the bad words from English but also consider the bad words from other languages.
         """),
-        Img(
-            src="path/to/sample_toxic_lines_removed.png",
-            alt="Sample documents with toxic lines",
+        view_data_static(
+            json.load(open("data/toxic_lines.json")),
+            "Sample documents with toxic lines",
         ),
         H3("3. Document-Level Filtering"),
         P("""
         In this section, we introduce all the quality signals that we have used to filter out low-quality documents.
-        Overview of all the quality signals that are used for filtering.
-        Similar to previous sections, we will present sample documents filtered out by the given quality signals. 
+        Overview of all the quality signals that are used for filtering."""),
+        view_data_static(
+            json.load(open("data/all_signals.json")),
+            "Overview of all the quality signals that are used for filtering",
+        ),
+        P("""Similar to previous sections, we will present sample documents filtered out by the given quality signals. 
         Most of these quality signals were initially introduced by Gopher [2] and subsequently adopted by later 
         studies ([3], [6], [4]). However, we observed that, despite following the same descriptions, the implementation 
         of each quality signal can vary significantly among different dataset pipelines, resulting in disparate