You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+48-8Lines changed: 48 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -148,6 +148,12 @@ This module should be run first to produce the dataset for the `osdash` dashboar
148
148
149
149
An example CSV list is included with this repository in [`input/OSH-repos.csv`](./input/OSH-repos.csv) and can be used out-of-the-box. This list was used to obtain the data visualised in the dashboard proof-of-concept [demo instance](https://opennextwp22.eu.pythonanywhere.com/).
150
150
151
+
The following is a screenshot of the example CSV list opened as a spreadsheet where columns and contents are visible:
152
+
153
+

154
+
155
+
***Figure 1.** Screenshot of input CSV list of repositories to mine opened as a spreadsheet.*
156
+
151
157
2. Obtain a [GitHub personal access token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token), which is a 40-character alphanumeric string in the form of `22x9w3npi629wgam5s6xxxen5abrozeb3db2mr42`. A GitHub account is required for this step. This token can be fed to `osmine` as: The environment variable `GH_TOKEN`; a path to a one-line pure-text file containing the string as a command line argument (see step 4); or directly in the configuration file (see step 3.).
152
158
153
159
3. Set up configuration file.
@@ -162,6 +168,12 @@ This module should be run first to produce the dataset for the `osdash` dashboar
162
168
python osmine
163
169
```
164
170
171
+
If successful, the first lines of terminal output should resemble the following:
172
+
173
+

174
+
175
+
***Figure 2.** First lines of output from data-mining script `osmine`.*
176
+
165
177
Unless provided elsewhere, these two command line arguments are the most important: **`-t`** - path to file containing GitHub access token;**`-r`** - path to CSV file containing list of repositories to query. Here is a full example assuming a pure-text file named `token` containing the GitHub token string has been placed in the base directory of `wp2.2_dev`:
166
178
167
179
```sh
@@ -176,12 +188,18 @@ This module should be run first to produce the dataset for the `osdash` dashboar
176
188
177
189
6. The `osmine` module will run, using its internal GitHub and Wikifactory modules ([`osmine/miner/GitHub.py`](./osmine/miner/GitHub.py) and [`osmine/miner/Wikifactory.py`](./osmine/miner/Wikifactory.py)) to query the provided list of repositories. At time of writing, the GitHub API allows filtering results by time to reduce network usage. `osmine` will use the "last mined" timestamp described above to only query fordata after that timestamp and append the results to previously saved data. During the run, you should see outputin your terminal that resembles the following:
178
190
179
-

191
+

180
192
181
-
***Figure 1.** Typical terminal output when running `osmine`.*
193
+
***Figure 3.** Typical terminal output when running `osmine`.*
182
194
183
195
7. Once data-mining is complete, the results will be saved in a [JSON](https://www.json.org/json-en.html) file with 4-spaces indentation (`mined_data.json`) contained in a compressed archive `data/mined_data.zip`. Currently this behaviour is hard-coded in the Python script [`osmine/postprocess/exporter.py`](./osmine/postprocess/exporter.py).
184
196
197
+
Since a JSON file is also a text file, the contents of `mined_data.json` could be directly inspected by opening it in any text editor. The following screenshot shows a section of the JSON-formatted data saved by the data-mining script `osmine` containing metadata on GitHub commits. For example, the timestamp of the commit, authorship information, and link to the commit are recorded.
198
+
199
+

200
+
201
+
***Figure 4.** Typical contents of exported data in JSON format from the data-mining script `osmine`.*
202
+
185
203
If no visualisation is needed, then there is no need to run anything else. The mined data file `data/mined_data.zip` can be used in other applications. See [Design notes](#design-notes) section on how the data file is structured.
186
204
187
205
### `osdash` dashboard module
@@ -198,21 +216,43 @@ On your server, the following steps will start a local test instance of the dash
198
216
python osdash
199
217
```
200
218
201
-
This will start the test server using mined-data from `osmine` saved in`data/mined_data.zip`. The initialisation process will take at least several seconds to serve the dashboard webpage.
219
+
This will start the test server using mined-data from `osmine` saved in`data/mined_data.zip`. The initialisation process will take at least several seconds to serve the dashboard webpage. The following is typical output in the terminal indicating the test server has been started:
220
+
221
+

222
+
223
+
***Figure 5.** Typical terminal output upon successful start of the test server of the proof-of-concept `osdash` online dashboard.*
202
224
203
225
2. Unless the default port number was changed (see [Install](#install) section), open the dashboard's URL in a web browser such as [Mozilla Firefox](https://www.mozilla.org/firefox/):
204
226
205
227
`http://127.0.0.1:21110` (no `https://`in this case)
206
228
207
229
Replace `21110` with another number if you changed the port. After a few seconds, the dashboard should be visible in your browser window. The following is a screenshot showing a typical session of the dashboard displaying metrics for the [`open-source-rover`](https://github.com/nasa-jpl/open-source-rover) project:
***Figure 2.** Typical screenshot of the proof-of-concept dashboard module `osdash`.*
233
+
***Figure 6.** Typical screenshot of the proof-of-concept dashboard module `osdash` opened in a web browser.*
212
234
213
235
3. There should already be statistics shown for a repository. If the example `input/OSH-list.csv` was used by `osmine`, then the dashboard should be showing information on the [Airpup](https://github.com/mathewlippincott/airpup-balloon) open source hardware aerial sensing platform.
214
236
215
-
4. The left side of the page is a sidebar forconfiguring which repository's information to show. Follow the numbered stepsin order to customise the view.
237
+
4. The left side of the page is a sidebar forconfiguring which repository's information to show. Follow the numbered stepsin order to customise the view. Specifically:
238
+
239
+
a. Begin by choosing an open source hardware project:
c. There is a slider with two draggable ends to filter the dataset for the selected repository by the desired timeframe. Click and drag them to specify the timeframe:
5. In this proof-of-concept version of the dashboard, the timeframe slider in the sidebar will constrain the data shown in the repository view to the right. This view has three components:
218
258
@@ -221,7 +261,7 @@ On your server, the following steps will start a local test instance of the dash
221
261
222
262

223
263
224
-
***Figure 3.** Hovering the mouse cursor over a dashboard plot will reveal interaction tools.*
264
+
***Figure 10.** Hovering the mouse cursor over a dashboard plot will reveal interaction tools.*
225
265
226
266
* A table showing the list of users (shown as usernames) who have contributed to this repository including their number of commits and the tickets they have participated in. Participation can be either opening or commenting on a ticket.
227
267
@@ -236,7 +276,7 @@ The essential sequence of operation for `osmine` and `osdash` has been described
0 commit comments