awswrangler.s3.list_objects

awswrangler.s3.list_objects(path: str, suffix: Optional[Union[str, List[str]]] = None, ignore_suffix: Optional[Union[str, List[str]]] = None, last_modified_begin: Optional[datetime] = None, last_modified_end: Optional[datetime] = None, ignore_empty: bool = False, chunked: bool = False, s3_additional_kwargs: Optional[Dict[str, Any]] = None, boto3_session: Optional[Session] = None) Union[List[str], Iterator[List[str]]]

List Amazon S3 objects from a prefix.

This function accepts Unix shell-style wildcards in the path argument. * (matches everything), ? (matches any single character), [seq] (matches any character in seq), [!seq] (matches any character not in seq). If you want to use a path which includes Unix shell-style wildcard characters (*, ?, []), you can use glob.escape(path) before passing the path to this function.

Note

The filter by last_modified begin last_modified end is applied after list all S3 files

Parameters
  • path (str) – S3 path (e.g. s3://bucket/prefix).

  • suffix (Union[str, List[str], None]) – Suffix or List of suffixes for filtering S3 keys.

  • ignore_suffix (Union[str, List[str], None]) – Suffix or List of suffixes for S3 keys to be ignored.

  • last_modified_begin – Filter the s3 files by the Last modified date of the object. The filter is applied only after list all s3 files.

  • last_modified_end (datetime, optional) – Filter the s3 files by the Last modified date of the object. The filter is applied only after list all s3 files.

  • ignore_empty (bool) – Ignore files with 0 bytes.

  • chunked (bool) – If True returns iterator, and a single list otherwise. False by default.

  • s3_additional_kwargs (Optional[Dict[str, Any]]) – Forwarded to botocore requests. e.g. s3_additional_kwargs={‘RequestPayer’: ‘requester’}

  • boto3_session (boto3.Session(), optional) – Boto3 Session. The default boto3 session will be used if boto3_session receive None.

Returns

List of objects paths.

Return type

Union[List[str], Iterator[List[str]]]

Examples

Using the default boto3 session

>>> import awswrangler as wr
>>> wr.s3.list_objects('s3://bucket/prefix')
['s3://bucket/prefix0', 's3://bucket/prefix1', 's3://bucket/prefix2']

Using a custom boto3 session

>>> import boto3
>>> import awswrangler as wr
>>> wr.s3.list_objects('s3://bucket/prefix', boto3_session=boto3.Session())
['s3://bucket/prefix0', 's3://bucket/prefix1', 's3://bucket/prefix2']