How to get the contents of a webpage but after running javascript?

Question

I am trying to download the contents of a web page using curl but it seems l am missing some content that l suspect to be generated by JavaScript. If the page runs on the browser l am able to get all the contents l require.

See http://superuser.com/questions/448514/command-line-browser-with-js-support — cas, Nov 03 '15 at 08:36
@Gilles I don't think it's a duplicate. This one here is more complex as it requires Javascript, which curl doesn't offer. On the other hand, I would agree it's a duplicate of the SU question that cas has suggested — Chris Davies, Nov 03 '15 at 22:37
@roaima Having Javascript was one of the points of that question as well. Both questions are asking for “curl” for the automation plus the trimmings of web browsers such as JS. — Gilles 'SO- stop being evil', Nov 03 '15 at 22:56
@Gilles I see nothing in your suggested duplicate that requires Javascript. (Someone did ask if it was required but there was no reply.) Furthermore the accepted answer offered curl, which mwiti here is already ruling out as inadequate. — Chris Davies, Nov 03 '15 at 23:15

score 2 · Answer 1 · edited Nov 03 '15 at 14:04

2

You can use PhantomJS to do it.

(From the link, PhantomJS is a headless WebKit scriptable with a JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.)

edited Nov 03 '15 at 14:04

Chris Davies

116,213
16
160
287

answered Nov 03 '15 at 13:58

Tam Nguyen

31
1

How to get the contents of a webpage but after running javascript?

1 Answers1