Skip to content

Commit 2f2cc6c

Browse files
JECSandJECSand
authored
Merge pull request #172 from JECSand/development
Development
2 parents 7e9f03f + 7f0b013 commit 2f2cc6c

File tree

10 files changed

+198
-52
lines changed

10 files changed

+198
-52
lines changed

‎CHANGES‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,3 +59,6 @@
5959
1.18 12/09/2023 -- Merged in branch from shaunpatterson to fix #164.
6060
1.19 12/12/2023 -- Refactored session management system to handle cookie and crumbs better.
6161
1.19 12/12/2023 -- Added fixes for #167, #166, #160.
62+
1.20 12/16/2023 -- Merged in pull request #171 from bjosun.
63+
1.20 12/17/2023 -- Added optional flat format output param on YahooFinancial class.
64+
1.20 12/17/2023 -- Added get_insight() and get_recommendations() methods.

‎README.rst‎

Lines changed: 59 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -16,16 +16,71 @@ A python module that returns stock, cryptocurrency, forex, mutual fund, commodit
1616
.. image:: https://static.pepy.tech/badge/yahoofinancials/week
1717
:target: https://pepy.tech/project/yahoofinancials
1818

19-
Current Version: v1.19
19+
Current Version: v1.20
2020

21-
Version Released: 12/12/2023
21+
Version Released: 12/17/2023
2222

2323
Report any bugs by opening an issue here: https://github.com/JECSand/yahoofinancials/issues
2424

2525
Overview
2626
--------
2727
A powerful financial data module used for pulling both fundamental and technical data from Yahoo Finance.
2828

29+
- New analytic methods in v1.20:
30+
- get_insights()
31+
- returns data for:
32+
- 'instrumentInfo'
33+
- 'companySnapshot'
34+
- 'recommendation'
35+
- 'sigDevs'
36+
- 'secReports'
37+
- get_recommendations()
38+
39+
- Example:
40+
41+
.. code-block:: python
42+
43+
print(YahooFinancials('C').get_recommendations())
44+
45+
- Example Output:
46+
47+
.. code-block:: javascript
48+
49+
{
50+
"C": [
51+
{
52+
"recommendedSymbols": [
53+
{
54+
"score": 0.239602,
55+
"symbol": "BAC"
56+
},
57+
{
58+
"score": 0.225134,
59+
"symbol": "JPM"
60+
},
61+
{
62+
"score": 0.167669,
63+
"symbol": "WFC"
64+
},
65+
{
66+
"score": 0.145864,
67+
"symbol": "GS"
68+
},
69+
{
70+
"score": 0.134071,
71+
"symbol": "F"
72+
}
73+
],
74+
"symbol": "C"
75+
}
76+
]
77+
}
78+
79+
- As of Version 1.20, YahooFinancials supports a new optional parameter called flat_format.
80+
- When `YahooFinancials(flat_format=True)`, financial statement data will return in a dict instead of a list. The keys of the dict will be the reporting dates.
81+
- Default is False, to ensure backwards compatibility.
82+
83+
2984
- As of Version 1.9, YahooFinancials supports optional parameters for asynchronous execution, proxies, and international requests.
3085

3186
.. code-block:: python
@@ -41,10 +96,6 @@ A powerful financial data module used for pulling both fundamental and technical
4196
balance_sheet_data_qt = yahoo_financials.get_financial_stmts('quarterly', 'balance')
4297
print(balance_sheet_data_qt)
4398
44-
- New methods in Version 1.13:
45-
- get_esg_score_data()
46-
47-
4899
Installation
49100
-------------
50101
- yahoofinancials runs on Python 3.7, 3.8, 3.9, 3.10, 3.11, and 3.12
@@ -91,7 +142,7 @@ Module Methods
91142
--------------
92143
- The financial data from all methods is returned as JSON.
93144
- You can run multiple symbols at once using an inputted array or run an individual symbol using an inputted string.
94-
- YahooFinancials works with Python 3.6, 3.7, 3.8, 3.9, 3.10, and 3.11 and runs on all operating systems. (Windows, Mac, Linux).
145+
- YahooFinancials works with Python 3.7, 3.8, 3.9, 3.10, 3.11 and 3.12 and runs on all operating systems. (Windows, Mac, Linux).
95146

96147
Featured Methods
97148
^^^^^^^^^^^^^^^^
@@ -134,6 +185,7 @@ Additional Module Methods
134185
- get_cost_of_revenue()
135186
- get_income_before_tax()
136187
- get_income_tax_expense()
188+
- get_esg_score_data()
137189
- get_gross_profit()
138190
- get_net_income_from_continuing_ops()
139191
- get_research_and_development()

‎setup.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,11 +10,11 @@
1010

1111
setup(
1212
name='yahoofinancials',
13-
version='1.19',
13+
version='1.20',
1414
description='A powerful financial data module used for pulling both fundamental and technical data from Yahoo Finance',
1515
long_description=long_description,
1616
url='https://github.com/JECSand/yahoofinancials',
17-
download_url='https://github.com/JECSand/yahoofinancials/archive/1.19.tar.gz',
17+
download_url='https://github.com/JECSand/yahoofinancials/archive/1.20.tar.gz',
1818
author='Connor Sanders',
1919
author_email='connor@exceleri.com',
2020
license='MIT',

‎test/test_yahoofinancials.py‎

Lines changed: 34 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
# YahooFinancials Unit Tests v1.19
2-
# Version Released: 12/12/2023
1+
# YahooFinancials Unit Tests v1.20
2+
# Version Released: 12/17/2023
33
# Author: Connor Sanders
4-
# Tested on Python 3.7, 3.8, 3.9, 3.10, and 3.11
4+
# Tested on Python 3.7, 3.8, 3.9, 3.10, 3.11, and 3.12
55
# Copyright (c) 2023 Connor Sanders <jecsand@pm.me>
66
# MIT License
77

@@ -42,6 +42,8 @@ def setUp(self):
4242
self.test_yf_treasuries_multi = yf(us_treasuries)
4343
self.test_yf_currencies = yf(currencies)
4444
self.test_yf_concurrent = yf(stocks, concurrent=True)
45+
self.test_yf_stock_flat = yf('C', flat_format=True)
46+
self.test_yf_stock_analytic = yf('WFC')
4547

4648
# Fundamentals Test
4749
def test_yf_fundamentals(self):
@@ -103,6 +105,35 @@ def test_yf_concurrency(self):
103105
result = check_fundamental(multi_all_statement_data_qt, 'all')
104106
self.assertEqual(result, True)
105107

108+
# Fundamentals in Flat Format Test
109+
def test_yf_fundamentals_flat(self):
110+
# Single stock test
111+
single_all_statement_data_qt = self.test_yf_stock_flat.get_financial_stmts('quarterly',
112+
['income', 'cash', 'balance'])
113+
if ((isinstance(single_all_statement_data_qt.get("incomeStatementHistoryQuarterly").get("C"), dict) and
114+
isinstance(single_all_statement_data_qt.get("balanceSheetHistoryQuarterly").get("C"), dict)) and
115+
isinstance(single_all_statement_data_qt.get("cashflowStatementHistoryQuarterly").get("C"), dict)):
116+
self.assertEqual(True, True)
117+
else:
118+
self.assertEqual(False, True)
119+
120+
# Analytic Methods Test
121+
def test_yf_analytic_methods(self):
122+
123+
# Get Insights
124+
out = self.test_yf_stock_analytic.get_insights()
125+
if out.get("WFC").get("instrumentInfo").get("technicalEvents").get("sector") == "Financial Services":
126+
self.assertEqual(True, True)
127+
else:
128+
self.assertEqual(False, True)
129+
130+
# Get Recommendations
131+
out = self.test_yf_stock_analytic.get_recommendations()
132+
if isinstance(out.get("WFC"), list):
133+
self.assertEqual(True, True)
134+
else:
135+
self.assertEqual(False, True)
136+
106137
# Extra Module Methods Test
107138
def test_yf_module_methods(self):
108139

‎yahoofinancials/cache.py‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -360,7 +360,6 @@ def initialise(self):
360360
continue
361361
self.initialised = 0 # failure
362362

363-
364363
def lookup(self, strategy):
365364
if self.dummy:
366365
return None

yahoofinancials/etl.py renamed to yahoofinancials/data.py

Lines changed: 46 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ def get_data(self, session, url, request_headers=None, params=None, proxy=None,
6060
return response
6161

6262

63-
class YahooFinanceETL(object):
63+
class YahooFinanceData(object):
6464

6565
def __init__(self, ticker, **kwargs):
6666
self.ticker = ticker.upper() if isinstance(ticker, str) else [t.upper() for t in ticker]
@@ -72,6 +72,7 @@ def __init__(self, ticker, **kwargs):
7272
self.timeout = kwargs.get("timeout", 30)
7373
self.proxies = kwargs.get("proxies")
7474
self.session = kwargs.pop("session", None)
75+
self.flat_format = kwargs.get("flat_format", False)
7576
self._cache = {}
7677

7778
# Minimum interval between Yahoo Finance requests for this instance
@@ -167,9 +168,15 @@ def _construct_url(self, symbol, config, params, freq, request_type):
167168
params.update({k: v['options'][request_type].get(freq)})
168169
elif k == "modules" and request_type in v['options']:
169170
params.update({k: request_type})
171+
elif k == "symbol":
172+
params.update({k: symbol.lower()})
170173
elif k not in params:
174+
if k == 'reportsCount' and v is None:
175+
continue
171176
params.update({k: v['default']})
172177
for k, v in _default_query_params.items(): # general defaults
178+
if k == 'reportsCount' and v is None:
179+
continue
173180
if k not in params:
174181
params.update({k: v})
175182
if params.get("type"):
@@ -183,6 +190,8 @@ def _construct_url(self, symbol, config, params, freq, request_type):
183190
for k, v in params.items():
184191
if k != "modules":
185192
url += "&" + k + "=" + str(v)
193+
elif params.get("symbol"):
194+
url += "?symbol=" + params.get("symbol")
186195
return url
187196

188197
# Private method to execute a web scrape request and decrypt the return
@@ -269,6 +278,11 @@ def _get_historical_data(self, url, config, tech_type, statement_type):
269278
data = self._cache[url]
270279
if tech_type == '' and statement_type in ["income", "balance", "cash"]:
271280
data = self._format_raw_fundamental_data(data)
281+
elif statement_type == 'analytic':
282+
data = data.get("result")
283+
if tech_type == "recommendations":
284+
if isinstance(data, list) and len(data) > 0:
285+
data[0].get("recommendedSymbols")
272286
else:
273287
data = self._format_raw_module_data(data, tech_type)
274288
return data
@@ -519,7 +533,9 @@ def _create_dict_ent(self, up_ticker, statement_type, tech_type, report_name, hi
519533
dict_ent = {}
520534
params = {}
521535
r_map = get_request_config(tech_type, REQUEST_MAP)
522-
r_cat = get_request_category(tech_type, self.YAHOO_FINANCIAL_TYPES, statement_type)
536+
r_cat = None
537+
if statement_type != 'analytic':
538+
r_cat = get_request_category(tech_type, self.YAHOO_FINANCIAL_TYPES, statement_type)
523539
YAHOO_URL = self._construct_url(
524540
up_ticker.lower(),
525541
r_map,
@@ -544,6 +560,17 @@ def _create_dict_ent(self, up_ticker, statement_type, tech_type, report_name, hi
544560
dict_ent = {up_ticker: re_data}
545561
return dict_ent
546562

563+
def _retry_create_dict_ent(self, up_ticker, statement_type, tech_type, report_name, hist_obj):
564+
i = 0
565+
while i < 250:
566+
try:
567+
out = self._create_dict_ent(up_ticker, statement_type, tech_type, report_name, hist_obj)
568+
return out
569+
except:
570+
time.sleep(random.randint(2, 10))
571+
i += 1
572+
continue
573+
547574
# Private method to return the stmt_id for the reformat_process
548575
def _get_stmt_id(self, statement_type, raw_data):
549576
stmt_id = ''
@@ -568,8 +595,22 @@ def _reformat_stmt_data_process(raw_data):
568595
else:
569596
return raw_data
570597

598+
# Private Method for the Flat Reformat Process
599+
@staticmethod
600+
def _reformat_stmt_data_process_flat(raw_data):
601+
final_data = {}
602+
if raw_data is not None:
603+
for date_key, data_item in raw_data.items():
604+
final_data.update({date_key: data_item})
605+
return final_data
606+
else:
607+
return raw_data
608+
571609
# Private Method to return subdict entry for the statement reformat process
572610
def _get_sub_dict_ent(self, ticker, raw_data):
611+
if self.flat_format:
612+
form_data_dict = self._reformat_stmt_data_process_flat(raw_data[ticker])
613+
return {ticker: form_data_dict}
573614
form_data_list = self._reformat_stmt_data_process(raw_data[ticker])
574615
return {ticker: form_data_list}
575616

@@ -581,17 +622,17 @@ def get_time_code(self, time_interval):
581622
# Public Method to get stock data
582623
def get_stock_data(self, statement_type='income', tech_type='', report_name='', hist_obj={}):
583624
data = {}
584-
if statement_type == 'income' and tech_type == '' and report_name == '': # temp, so this method doesn't return nulls
625+
if statement_type == 'income' and tech_type == '' and report_name == '': # temp, so this method doesn't return nulls
585626
statement_type = 'profile'
586627
tech_type = 'assetProfile'
587628
report_name = 'assetProfile'
588629
if isinstance(self.ticker, str):
589-
dict_ent = self._create_dict_ent(self.ticker, statement_type, tech_type, report_name, hist_obj)
630+
dict_ent = self._retry_create_dict_ent(self.ticker, statement_type, tech_type, report_name, hist_obj)
590631
data.update(dict_ent)
591632
else:
592633
if self.concurrent:
593634
with Pool(self._get_worker_count()) as pool:
594-
dict_ents = pool.map(partial(self._create_dict_ent,
635+
dict_ents = pool.map(partial(self._retry_create_dict_ent,
595636
statement_type=statement_type,
596637
tech_type=tech_type,
597638
report_name=report_name,

‎yahoofinancials/maps.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2375,6 +2375,19 @@
23752375
"padTimeSeries": {"required": False, "default": False},
23762376
},
23772377
},
2378+
"insights": {
2379+
"path": "https://query1.finance.yahoo.com/ws/insights/v2/finance/insights",
2380+
"response_field": "finance",
2381+
"request": {
2382+
"symbol": {"required": True, "default": None},
2383+
"reportsCount": {"required": False, "default": None},
2384+
},
2385+
},
2386+
"recommendations": {
2387+
"path": "https://query1.finance.yahoo.com/v6/finance/recommendationsbysymbol/{symbol}",
2388+
"response_field": "finance",
2389+
"request": {},
2390+
},
23782391
}
23792392

23802393
USER_AGENTS = [

0 commit comments

Comments
 (0)