awswrangler.s3.to_json¶
-
awswrangler.s3.
to_json
(df: pandas.core.frame.DataFrame, path: str, boto3_session: Optional[boto3.session.Session] = None, s3_additional_kwargs: Optional[Dict[str, Any]] = None, use_threads: bool = True, **pandas_kwargs: Any) → List[str]¶ Write JSON file on Amazon S3.
Note
In case of use_threads=True the number of threads that will be spawned will be gotten from os.cpu_count().
Note
Compression: The minimum acceptable version to achive it is Pandas 1.2.0 that requires Python >= 3.7.1.
- Parameters
df (pandas.DataFrame) – Pandas DataFrame https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html
path (str) – Amazon S3 path (e.g. s3://bucket/filename.csv).
boto3_session (boto3.Session(), optional) – Boto3 Session. The default boto3 Session will be used if boto3_session receive None.
s3_additional_kwargs (Optional[Dict[str, Any]]) – Forward to botocore requests. Valid parameters: “ACL”, “Metadata”, “ServerSideEncryption”, “StorageClass”, “SSECustomerAlgorithm”, “SSECustomerKey”, “SSEKMSKeyId”, “SSEKMSEncryptionContext”, “Tagging”. e.g. s3_additional_kwargs={‘ServerSideEncryption’: ‘aws:kms’, ‘SSEKMSKeyId’: ‘YOUR_KMS_KEY_ARN’}
use_threads (bool) – True to enable concurrent requests, False to disable multiple threads. If enabled os.cpu_count() will be used as the max number of threads.
pandas_kwargs – KEYWORD arguments forwarded to pandas.DataFrame.to_json(). You can NOT pass pandas_kwargs explicit, just add valid Pandas arguments in the function call and Wrangler will accept it. e.g. wr.s3.to_json(df, path, lines=True, date_format=’iso’) https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_json.html
- Returns
List of written files.
- Return type
List[str]
Examples
Writing JSON file
>>> import awswrangler as wr >>> import pandas as pd >>> wr.s3.to_json( ... df=pd.DataFrame({'col': [1, 2, 3]}), ... path='s3://bucket/filename.json', ... )
Writing JSON file using pandas_kwargs
>>> import awswrangler as wr >>> import pandas as pd >>> wr.s3.to_json( ... df=pd.DataFrame({'col': [1, 2, 3]}), ... path='s3://bucket/filename.json', ... lines=True, ... date_format='iso' ... )
Writing CSV file encrypted with a KMS key
>>> import awswrangler as wr >>> import pandas as pd >>> wr.s3.to_json( ... df=pd.DataFrame({'col': [1, 2, 3]}), ... path='s3://bucket/filename.json', ... s3_additional_kwargs={ ... 'ServerSideEncryption': 'aws:kms', ... 'SSEKMSKeyId': 'YOUR_KMS_KEY_ARN' ... } ... )