Skip to content

Sourcery refactored main branch #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sourcery-ai wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Sourcery refactored main branch #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion NotifyMe/NM_analytics_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ def plot_most_frequent_wait():


def plot_most_freq_search_term(n=10):
title='<b> Top '+str(n)+' Most Frequent Search Terms </b>'
title = f'<b> Top {str(n)} Most Frequent Search Terms </b>'

@sourcery-ai sourcery-ai Bot May 25, 2022

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function plot_most_freq_search_term refactored with the following changes:

fig = px.bar(df_imp_keywords[:n], x="keywords",y='Users_Count', title=title, color= 'status', color_discrete_sequence=px.colors.qualitative.Vivid).update_xaxes(categoryorder="total descending")
fig.update_layout( title={'y':0.9, 'x':0.5,'xanchor': 'center','yanchor': 'top'},
titlefont=dict(size =28, color='black', family='Old Standard TT, serif'),
Expand Down
19 changes: 9 additions & 10 deletions NotifyMe/Notifyme_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,16 +94,15 @@ def retrieve_search_to_df(keyword):
rsp = json.loads(rsp.text)
if rsp['hits']['total']==0:
return 'Nothing matching yet'
else:
df_search=pd.DataFrame.from_dict(rsp['hits']['hits'])
df_search=pd.concat([df_search['_source'].apply(pd.Series)['pennsieve'].apply(pd.Series)[['identifier']],
df_search[['_id']],#df_search[['_id','_score']],
df_search['_source'].apply(pd.Series)['item'].apply(pd.Series)[['name','description']],
df_search['_source'].apply(pd.Series)['item'].apply(pd.Series)['published'].apply(pd.Series)['status'],
], axis=1)
df_search['link']= 'https://sparc.science/datasets/'+df_search['identifier']
df_search=df_search[['_id','name','description','status','link']]
return df_search
df_search=pd.DataFrame.from_dict(rsp['hits']['hits'])
df_search=pd.concat([df_search['_source'].apply(pd.Series)['pennsieve'].apply(pd.Series)[['identifier']],
df_search[['_id']],#df_search[['_id','_score']],
df_search['_source'].apply(pd.Series)['item'].apply(pd.Series)[['name','description']],
df_search['_source'].apply(pd.Series)['item'].apply(pd.Series)['published'].apply(pd.Series)['status'],
], axis=1)
df_search['link']= 'https://sparc.science/datasets/'+df_search['identifier']
df_search=df_search[['_id','name','description','status','link']]
return df_search
Comment on lines -97 to +105

@sourcery-ai sourcery-ai Bot May 25, 2022

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function retrieve_search_to_df refactored with the following changes:


### Sending Email with the results
def send_email_alert_withtable(emaillist, keyword):
Expand Down
54 changes: 30 additions & 24 deletions aqua_docker/app/utils/helper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,10 @@ def loadMain():
"""
fileName = os.path.join(os.path.dirname(__file__), 'main.html')
try:
f = open(fileName, "r")
htmlText = f.read()
f.close()
with open(fileName, "r") as f:
htmlText = f.read()
except:
return html("We are sorry,there was an error ..., %s is not found" % fileName)
return html(f"We are sorry,there was an error ..., {fileName} is not found")
Comment on lines -21 to +24

@sourcery-ai sourcery-ai Bot May 25, 2022

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function loadMain refactored with the following changes:

return html(htmlText)


Expand Down Expand Up @@ -59,7 +58,8 @@ def getSearch(request):
"""
# initialise query run
summary = {}
url = 'https://scicrunch.org/api/1/elastic/SPARC_PortalDatasets_pr/_search?api_key='+api_key
url = f'https://scicrunch.org/api/1/elastic/SPARC_PortalDatasets_pr/_search?api_key={api_key}'

@sourcery-ai sourcery-ai Bot May 25, 2022

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function getSearch refactored with the following changes:

size = '200'
includes = ["item.keywords.keyword",
"item.name",
Expand Down Expand Up @@ -131,7 +131,7 @@ def getSearch(request):
## extract from contributors
firsts = __find('_source.contributors.first.name', hit)
lasts = __find('_source.contributors.last.name', hit)
for name in [first+' '+last for first, last in (zip(firsts, lasts))]:
for name in [f'{first} {last}' for first, last in (zip(firsts, lasts))]:
if name not in summary['filters']['authors']:
summary['filters']['authors'][name] = []
summary['filters']['authors'][name] += [idx]
Expand All @@ -149,22 +149,28 @@ def getSearch(request):
names += [str(__find('_source.item.name', hit))]
srtNames += [str(idx)]
# set hit
ht = {'url': 'https://sparc.science/datasets/'+idx,
'banner': __find('_source.pennsieve.banner.uri', hit),
'_id': hit['_id'],
'_score': hit['_score'],
'firstPublishedAt': __find('_source.pennsieve.firstPublishedAt.timestamp', hit),
'updatedAt': __find('_source.pennsieve.updatedAt.timestamp', hit),
'name': __find('_source.item.name', hit),
'description': __find('_source.item.description', hit),
'readme': __find('_source.item.readme.description', hit),
'samples': __find('_source.item.statistics.samples.count', hit),
'subjects': __find('_source.item.statistics.subjects.count', hit),
'anatomy': __find('_source.anatomy.organ.name', hit),
'organisms': __find('_source.organisms.primary.species.originalName', hit),
'publication': __find('_source.item.published.boolean', hit),
'techniques': __find('_source.item.techniques.keyword', hit),
}
ht = {
'url': f'https://sparc.science/datasets/{idx}',
'banner': __find('_source.pennsieve.banner.uri', hit),
'_id': hit['_id'],
'_score': hit['_score'],
'firstPublishedAt': __find(
'_source.pennsieve.firstPublishedAt.timestamp', hit
),
'updatedAt': __find('_source.pennsieve.updatedAt.timestamp', hit),
'name': __find('_source.item.name', hit),
'description': __find('_source.item.description', hit),
'readme': __find('_source.item.readme.description', hit),
'samples': __find('_source.item.statistics.samples.count', hit),
'subjects': __find('_source.item.statistics.subjects.count', hit),
'anatomy': __find('_source.anatomy.organ.name', hit),
'organisms': __find(
'_source.organisms.primary.species.originalName', hit
),
'publication': __find('_source.item.published.boolean', hit),
'techniques': __find('_source.item.techniques.keyword', hit),
}

if 'highlight' in hit:
ht['highlight'] = {}
for type, value in hit['highlight'].items():
Expand Down Expand Up @@ -199,7 +205,7 @@ def __getSuggestions(query, limit):
# get possible correction
correction = __getCorrection(query)
# get suggestion from SciGraph
url = 'https://scicrunch.org/api/1/scigraph/vocabulary/suggestions/'+query
url = f'https://scicrunch.org/api/1/scigraph/vocabulary/suggestions/{query}'
Comment on lines -202 to +208

@sourcery-ai sourcery-ai Bot May 25, 2022

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function __getSuggestions refactored with the following changes:

params = {'api_key': api_key, 'limit': limit}
rsp = requests.get(url, params=params)
# return the correction and suggestions
Expand All @@ -223,7 +229,7 @@ def __getAutoComplete_sc(query, limit, verbose):
"""
To get autocomplete as a list data type.
"""
url = 'https://scicrunch.org/api/1/scigraph/vocabulary/autocomplete/'+query
url = f'https://scicrunch.org/api/1/scigraph/vocabulary/autocomplete/{query}'
Comment on lines -226 to +232

@sourcery-ai sourcery-ai Bot May 25, 2022

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function __getAutoComplete_sc refactored with the following changes:

params = {'api_key': api_key, 'limit': limit, 'searchSynonyms': 'true',
'searchAbbreviations': 'false', 'searchAcronyms': 'false',
'includeDeprecated': 'false'}
Expand Down
21 changes: 10 additions & 11 deletions aqua_docker/app/utils/notifyme_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,17 +89,16 @@ def retrieve_search_to_df(keyword):
rsp = json.loads(rsp.text)
if rsp['hits']['total']==0:
return 'Nothing matching yet'
else:
df_search=pd.DataFrame.from_dict(rsp['hits']['hits'])
df_search=pd.concat([
df_search['_source'].apply(pd.Series)['pennsieve'].apply(pd.Series)[['identifier']],
df_search[['_id','_score']],
df_search['_source'].apply(pd.Series)['item'].apply(pd.Series)[['name','description']],
df_search['_source'].apply(pd.Series)['item'].apply(pd.Series)['published'].apply(pd.Series)['status'],
], axis=1)
df_search['link']= 'https://sparc.science/datasets/'+df_search['identifier']

return df_search
df_search=pd.DataFrame.from_dict(rsp['hits']['hits'])
df_search=pd.concat([
df_search['_source'].apply(pd.Series)['pennsieve'].apply(pd.Series)[['identifier']],
df_search[['_id','_score']],
df_search['_source'].apply(pd.Series)['item'].apply(pd.Series)[['name','description']],
df_search['_source'].apply(pd.Series)['item'].apply(pd.Series)['published'].apply(pd.Series)['status'],
], axis=1)
df_search['link']= 'https://sparc.science/datasets/'+df_search['identifier']

return df_search
Comment on lines -92 to +101

@sourcery-ai sourcery-ai Bot May 25, 2022

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function retrieve_search_to_df refactored with the following changes:

# Sending Email with the results
def send_email_alert_withtable(emaillist, keyword):
msg = MIMEMultipart()
Expand Down