# scrape-it-core
[![Support me on Patreon][badge_patreon]][patreon] [![Buy me a book][badge_amazon]][amazon] [![PayPal][badge_paypal_donate]][paypal-donations] [](https://github.com/IonicaBizau/ama) [](https://www.npmjs.com/package/scrape-it-core) [](https://www.npmjs.com/package/scrape-it-core) [](https://www.codementor.io/johnnyb?utm_source=github&utm_medium=button&utm_term=johnnyb&utm_campaign=github)
> The core scraping functionality of scrape-it.
## :cloud: Installation
```sh
# Using npm
npm install --save scrape-it-core
# Using yarn
yarn add scrape-it-core
```
## :clipboard: Example
```js
const scrapeIt = require("scrape-it-core")
// Promise interface
const data = scrapeIt(`
Hello World
Lorem ipsum dolor sit amet...
`, {
title: ".heading"
, desc: "p"
, avatar: {
selector: "img"
, attr: "src"
}
, items: {
listItem: "ul > li"
, data: {
content: {
how: "text"
}
}
}
})
console.log(data)
// { title: 'Hello World',
// desc: 'Lorem ipsum dolor sit amet...',
// avatar: 'test.jpg',
// items:
// [ { content: '1' },
// { content: '2' },
// { content: '3' },
// { content: '4' },
// { content: '5' },
// { content: '6' } ] }
```
## :question: Get Help
There are few ways to get help:
1. Please [post questions on Stack Overflow](https://stackoverflow.com/questions/ask). You can open issues with questions, as long you add a link to your Stack Overflow question.
2. For bug reports and feature requests, open issues. :bug:
3. For direct and quick help, you can [use Codementor](https://www.codementor.io/johnnyb). :rocket:
## :memo: Documentation
### `scrapeItCore($, opts)`
The core scraping functionality of `scrape-it`.
Scrapes the data in the provided element.
For the format of the selector, please refer to the [Selectors section of the Cheerio library](https://github.com/cheeriojs/cheerio#-selector-context-root-)
#### Params
- **Cheerio|String** `$`: The input element or the HTML as a string.
- **Object** `opts`: An object containing the scraping information.
If you want to scrape a list, you have to use the `listItem` selector:
- `listItem` (String): The list item selector.
- `data` (Object): The fields to include in the list objects:
- `` (Object|String): The selector or an object containing:
- `selector` (String): The selector.
- `convert` (Function): An optional function to change the value.
- `how` (Function|String): A function or function name to access the
value.
- `attr` (String): If provided, the value will be taken based on
the attribute name.
- `trim` (Boolean): If `false`, the value will *not* be trimmed
(default: `true`).
- `closest` (String): If provided, returns the first ancestor of
the given element.
- `eq` (Number): If provided, it will select the *nth* element.
- `texteq` (Number): If provided, it will select the *nth* direct text child.
Deep text child selection is not possible yet.
Overwrites the `how` key.
- `listItem` (Object): An object, keeping the recursive schema of
the `listItem` object. This can be used to create nested lists.
**Example**:
```js
{
articles: {
listItem: ".article"
, data: {
createdAt: {
selector: ".date"
, convert: x => new Date(x)
}
, title: "a.article-title"
, tags: {
listItem: ".tags > span"
}
, content: {
selector: ".article-content"
, how: "html"
}
, traverseOtherNode: {
selector: ".upperNode"
, closest: "div"
, convert: x => x.length
}
}
}
}
```
If you want to collect specific data from the page, just use the same
schema used for the `data` field.
**Example**:
```js
{
title: ".header h1"
, desc: ".header h2"
, avatar: {
selector: ".header img"
, attr: "src"
}
}
```
#### Return
- **Object** The scraped data.
## :yum: How to contribute
Have an idea? Found a bug? See [how to contribute][contributing].
## :sparkling_heart: Support my projects
I open-source almost everything I can, and I try to reply to everyone needing help using these projects. Obviously,
this takes time. You can integrate and use these projects in your applications *for free*! You can even change the source code and redistribute (even resell it).
However, if you get some profit from this or just want to encourage me to continue creating stuff, there are few ways you can do it:
- Starring and sharing the projects you like :rocket:
- [![Buy me a book][badge_amazon]][amazon]—I love books! I will remember you after years if you buy me one. :grin: :book:
- [![PayPal][badge_paypal]][paypal-donations]—You can make one-time donations via PayPal. I'll probably buy a ~~coffee~~ tea. :tea:
- [![Support me on Patreon][badge_patreon]][patreon]—Set up a recurring monthly donation and you will get interesting news about what I'm doing (things that I don't share with everyone).
- **Bitcoin**—You can send me bitcoins at this address (or scanning the code below): `1P9BRsmazNQcuyTxEqveUsnf5CERdq35V6`
Thanks! :heart:
## :scroll: License
[MIT][license] © [Ionică Bizău][website]
[license]: /LICENSE
[website]: https://ionicabizau.net
[contributing]: /CONTRIBUTING.md
[docs]: /DOCUMENTATION.md
[badge_patreon]: https://ionicabizau.github.io/badges/patreon.svg
[badge_amazon]: https://ionicabizau.github.io/badges/amazon.svg
[badge_paypal]: https://ionicabizau.github.io/badges/paypal.svg
[badge_paypal_donate]: https://ionicabizau.github.io/badges/paypal_donate.svg
[patreon]: https://www.patreon.com/ionicabizau
[amazon]: http://amzn.eu/hRo9sIZ
[paypal-donations]: https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=RVXDDLKKLQRJW