Ipython notebook page

[sahilshekhawat]

Firstly, this in no way is completed.This is just the basic structure on which we can inplement ipython notebook.I have actually the main thing to write. I have done several things because i felt them to be good so please help me out where you feel that something needs to be changed. Following are the points which i have applied or thought:
1)I have used 'r' variable from views.input and parsed it with notebook.py file which have all the formats predefined and i just childishly changed their values and appended.
2)That result variable is passed in the url. i know that there is a maximum length of url but i thought it will large enough to not exceed.
3)I didn't stored the result into the databases because it is not free and if i can use something else why should i use them.
4)Plotting and cards are to be implemented and after that this will be very cool because the possibilities are just endless
5)i have a view which renders the json and i have given the link to the nbviewer which renders out json.

Demo: http://sympygamma-pr-32.appspot.com

[asmeurer]

Why this change? Accidental?

[sahilshekhawat]

When i updated my submodules after cloning the repo, docutils also got updated and it showed untracked contents. so i had to commit those changes into docutils and after that commit the modified docutils into my repo.

[lidavidm]

I think the untracked contents are due to .pyc files? In any case, it shows for me too but I don't think we need to commit these changes.

[asmeurer]

Oh yeah, I see that all the time. We should use some fork that has a .gitignore.

[sahilshekhawat]

you are right, Docutils dont have .gitignore.

[lidavidm]

In that case, just revert this change.

[sahilshekhawat]

i have rebased that commit.

[lidavidm]

Overall, this is a good start. The notebook JSON is being generated correctly (I think), but there are little issues preventing it from working. I've put up a demo but it raises an exception.

[sahilshekhawat]

Thank you David for pointing out these issues to me. I will be commiting these changes along with the ability to create a gist and create a complete JSON.
Right now i am working on creating Plots into notebook.
Thanks a lot

[asmeurer]

Let's definitely not store things in the database. The database size is limited, and I really don't want to have to wipe everyone's notebooks when it fills up.

[sahilshekhawat]

ok! we will not use databases.. thats why we are trying to export them to GIST.

[sahilshekhawat]

Have not added the codes on which i am working ( exporting notebook to GIST and plotting)but have cleaned up the code a bit.

[sahilshekhawat]

@lidavidm Can we merge this now?...or if we want to test it first..can you please restart its build?

[sahilshekhawat]

@lidavidm Can we merge it?

[lidavidm]

This might be better of with a regex or with an actual HTML parser. If the output ever changes, this would break.

[sahilshekhawat]

I am already using an htmlparser (see top) and i think "replace" should do the trick, also I dont want to increase the complexity of the code. I know its silly to discuss these little points, but am I right?

[lidavidm]

Hmm. I'm trying to determine the intent of this section of code. It is trying to extract the input and output sections from a multiple-result card, correct? I think there are better ways to handle this. See https://gist.github.com/lidavidm/37a0a27582805872334d - it just uses the HTML parser and extracts both the input expression and output LaTeX.

[ashutoshsaboo]

@lidavidm @asmeurer @sahilshekhawat

So, I was looking into this, trying to take this further, and add many more features to this.

But, what I feel that, the Parser that has been used in notebook.py takes data from the cards displayed on SymPy Gamma right?

But the thing is that, the card data, i.e, the output doesn't load instantaneously as soon as SymPy Gamma loads. If you notice, in the bottom of your browser you can notice that, additional requests (as well as LaTeX needs to format the output as well, which is also done after SymPy Gamma loads) are sent after SymPy Gamma loads, to display the output. Only the input, and headings of all the cards load instantaneously as SymPy Gamma loads. Now, the thing that is happening is, since only the input and headings of the cards load as, SymPy Gamma loads, then the notebook.py file adds only the the input and the headings of the cards to the iPython notebook, and doesn't add output to the iPython notebook, since it loads later after SymPy Gamma loads.

Now, if we change this line - Line No 96 in notebook.py to - card_json_output = card_json['value'] , then the iPython notebook loads the output of the cards only (still it doesn't display the main output, which is different from the output of the cards), but it doesn't display them in LaTeX, since in logic.py - Last 2 lines -:

        return {
            'value': repr(result),
            'output': card.format_output(result, mathjax_latex)
        }

These lines show that, card_json['value'] returns only repr(result) while, card_json['output'] returns the LaTeX output, which instead loads later than when SymPy Gamma loads, and hence it doesn't get displayed in iPython notebook.

So, how do I tansform the card_json['value'] to LaTeX output? And also, as an alternative, how do I ensure that notebook.py runs after the complete results page gets loaded, with the output as well?

Another thing, that I am currently running SymPy Gamma locally on my localhost. Is that causing this issue?

@lidavidm @asmeurer @sahilshekhawat @certik Could you help me out on this?

[ashutoshsaboo]

Or how about this, We can scrape the contents of the card.html files - for the corresponding cards? i.e, we can use urllib2 to open the respective url's of the cards, and then extract the data out of them, and then we can add them to the iPython notebook.

This will also help us, since we won't be recomputing the output, so it'll efficient as well. And additionally, as @lidavidm suggested above, it won't break, even if the output changes a bit, since only the HTML Tag needs to change.

So, how about if we do this? @lidavidm @certik @asmeurer @sahilshekhawat

Your opinions on this one? Please help me out with this, as I'm not see the outputs of the cards in the iPython Notebook. 😄

[lidavidm]

logic.py does return the formatted output. It's right there: card.format_output. Why not use that?

I would not scrape the html files. You will have to write some code to do what you want within Gamma, but it's entirely doable.

[ashutoshsaboo]

@lidavidm Yes, I agree with you. I tried scraping the files, and it was a lot cumbersome. But, as I told, if I edit, the card_json['output'] to card_json['value'] , then how can I call the card.format_output function in notebook.py , since there is no object of class card right?

[ashutoshsaboo]

@lidavidm Could you help me on this^? @asmeurer @certik

[lidavidm]

I'm not sure I understand the issue here. Why must we use card_json['value']? The data is already there, and as I discussed with the original author, all that was needed was a better way to format the output. If you want, it should be trivial to modify eval_card in logic.py to return the original value in addition to the other two values, so that you can do the formatting yourself. Or, can't we insert HTML into an IPython notebook? Or you can modify eval_card so that instead of hardcoding mathjax_latex, it accepts a different formatting function that does what we want.

[ashutoshsaboo]

That is the problem happening.

If we keep car_json['output'] , then as I explained above, the output of cards load later than the input and headings of cards , hence notebook.py only saves the input and the headings of cards in the iPython Notebook, since the output of the cards haven't loaded yet, and they load later, since they need to be formatted into LaTeX. That's what card_json['value'] returns me the actual value of the output of the card without formatting. I didn't completely get what you are saying though, since what I think is that the data is not already there in the iPython notebook and that's the issue.

Yes, we can use HTML in an iPython Notebook - https://stackoverflow.com/questions/25698448/how-to-embed-html-into-ipython-output .

Also, how would you format the code into LaTeX? Could you suggest any detailing on that? @lidavidm

[lidavidm]

Is there a demo of this you can show me? It's been long enough that I don't recall how it works anymore.

Also, if you're talking about MathJax rendering the equations - that is unavoidable.

[ashutoshsaboo]

@lidavidm Actually, I am trying to deploy it on the appspot. I was using the instructions mentioned here - https://github.com/sympy/sympy_gamma . But then, how do I come to know about my App Engine Application version, listed about in the first step in the link above?

If you can help me with deploying on the appspot, then I'll post the link with the live demo.

[lidavidm]

You can use whatever version number you want, it's not important for a test. After you deploy go into the App Engine settings panel and set the default version number to the one you deployed. You don't need to tag.

[ashutoshsaboo]

@lidavidm Ohh okay. Where is the App Engine Settings panel? Also, where is sympy_gamma dashboard at appspot.com located? Links maybe for both?

[lidavidm]

Those instructions are for the official application. They should probably be updated to be clearer. You need to change the application name to your own, since you don't have permissions to deploy on the official application. The dashboard is https://appengine.google.com/, that's the App Engine panel I'm talking about. Create a project there, and use that name in your own app.yaml.

[ashutoshsaboo]

How do I handle this error? @lidavidm -:

(sympygamma) ashutoshsaboo@ASHUTOSH-PC:~/sympy_gamma$ python deploy.py --generate-only --generate-production
Generating test configuration.
ERROR: Must provide --generate-test VERSION if not running under Travis.

[lidavidm]

Follow the instructions. Use --generate-test not --generate-production, again, you're making a test version.

[ashutoshsaboo]

I tried that earlier. That also gave this error-:

(sympygamma) ashutoshsaboo@ASHUTOSH-PC:~/sympy_gamma$ python deploy.py --generate-only --generate-test
Generating test configuration.
ERROR: Must provide --generate-test VERSION if not running under Travis.

[lidavidm]

Again...follow the instructions. It's asking for the application version number, which you can fill in as you please, because it's just a test version.

[lidavidm]

We should probably revisit the documentation, though. Glancing through App Engine they've changed things, and we should document how to set up your own test instance.

[ashutoshsaboo]

Ohh yes, I'll issue a PR later for the same. Meanwhile, I uploaded my SymPy Gamma to the Appspot, but no results seem to get displayed. For instance here - https://sympy-gamma-testing.appspot.com/input/?i=tan%281%20%2B%20x%29 . Why is that so?

[lidavidm]

You enabled HTTPS but the assets are loaded over HTTP, so most browsers will block them, and so the rest of the page won't run.

[ashutoshsaboo]

Ohh now the iPython notebook seems to work fine. But, if you go here - http://sympy-gamma-testing.appspot.com/input/?i=integral%28x**2%29 , and then if you view the iPython notebook online, then it seems to appear fine. But if you download the file, and then try to view it locally on your system by jupyter-notebook , then no output will be visible. Why does that happen? @lidavidm

That's the thing that was happening to me. Since I was on localhost, hence I couldn't view it online definitely. So, I was viewing the results by downloading the files, and every time I couldn't view the results. Hence I was in trouble starting to find ways to fix this.

So, this is an issue in this right?

[ashutoshsaboo]

@lidavidm But interestingly even if you download the iPython notebook file from here- http://nbviewer.jupyter.org/url/sympy-gamma-testing.appspot.com/export_notebook//%3Fi%3Dintegral%2528x%252A%252A2%2529 , then also the output won't be visible if you try to open on your local system.

So, seems to be some problem with jupyter-notebook only, and not an issue in this right? @lidavidm

[lidavidm]

@ashutoshsaboo All your demos are not working, what's the status of this?

Specifically - the assets are still served via HTTP, so the site is broken when visiting with HTTPS, and Gamma seems to have an internal error when making the IPython Notebook.

[ashutoshsaboo]

@lidavidm Ohh yes I see. But what's the problem happening? Should I re-push my SymPy Gamma directory to google app engine and try again? Or what should I do?

But if you check this link without HTTPS - http://sympy-gamma-testing.appspot.com/input/?i=integral%28x**2%29 . Then it seems to work fine. Hence, definitely there's no issue with my SymPy Gamma. So what's the issue specifically? Is this some issue with the Google App Engine itself? Or what?

Additionally, the iPython notebook feature (with links at the bottom of the page - url mentioned above) - doesn't seem to work now? Why is that happening suddenly when it was all fine some time ago, and also that I haven't pushed any new change to GAE since then?

@asmeurer

[lidavidm]

For testing, I think we should enable debug mode, so we can see what the exception is.

I've explained the issue with HTTPS - the dependencies are not loaded with HTTPS, but the main page is, so your browser refuses to load the dependencies. We should just load all assets with HTTPS if we can, else we should use protocol-relative URLs.