Skip to content

Commit

Permalink
Merge pull request #3524 from IQSS/develop
Browse files Browse the repository at this point in the history
v4.6
  • Loading branch information
kcondon authored Dec 9, 2016
2 parents 7e5e8c9 + 52e9ec3 commit c913662
Show file tree
Hide file tree
Showing 158 changed files with 10,214 additions and 5,126 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Dataverse
==========

Dataverse is an open source web application for sharing, citing, analyzing, and preserving research data (developed by the [Data Science team] (http://datascience.iq.harvard.edu/about-dataverse) at the [Institute for Quantitative Social Science] (http://iq.harvard.edu/)).
Dataverse is an open source web application for sharing, citing, analyzing, and preserving research data (developed by the [Data Science and Products team] (http://www.iq.harvard.edu/people/people/data-science-products) at the [Institute for Quantitative Social Science] (http://iq.harvard.edu/)).

Institutions and organizations can choose to install the Dataverse software for their own use.
In this case, the institution will be responsible for maintaining the application; installing upgrades,
Expand Down
10 changes: 5 additions & 5 deletions conf/httpd/conf.d/dataverse.conf
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ ProxyPassMatch ^/error-documents !
# pass everything else to Glassfish
ProxyPass / ajp://localhost:8009/

<Location /shib.xhtml>
AuthType shibboleth
ShibRequestSetting requireSession 1
require valid-user
</Location>
#<Location /shib.xhtml>
# AuthType shibboleth
# ShibRequestSetting requireSession 1
# require valid-user
#</Location>

ErrorDocument 503 /error-documents/503.html
Alias /error-documents /var/www/dataverse/error-documents
Expand Down
2 changes: 2 additions & 0 deletions conf/solr/4.6.0/schema.xml
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,8 @@
<field name="unf" type="string" stored="true" indexed="true" multiValued="false"/>
<field name="fileSizeInBytes" type="long" stored="true" indexed="true" multiValued="false"/>
<field name="fileMd5" type="string" stored="true" indexed="true" multiValued="false"/>
<field name="fileChecksumType" type="string" stored="true" indexed="true" multiValued="false"/>
<field name="fileChecksumValue" type="string" stored="true" indexed="true" multiValued="false"/>
<field name="fileContentType" type="string" stored="true" indexed="true" multiValued="false"/>
<field name="deaccessionReason" type="string" stored="true" indexed="false" multiValued="false"/>

Expand Down
15 changes: 15 additions & 0 deletions doc/sphinx-guides/source/_static/api/dataverse-complete.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"name": "Scientific Research",
"alias": "science",
"dataverseContacts": [
{
"contactEmail": "[email protected]"
},
{
"contactEmail": "[email protected]"
}
],
"affiliation": "Scientific Research University",
"description": "We do all the science.",
"dataverseType": "LABORATORY"
}
9 changes: 9 additions & 0 deletions doc/sphinx-guides/source/_static/api/dataverse-minimal.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"name": "Scientific Research",
"alias": "science",
"dataverseContacts": [
{
"contactEmail": "[email protected]"
}
]
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
<?xml version='1.0' encoding='UTF-8' ?>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<title>Harvard Dataverse</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta http-equiv="Content-Language" content="en" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="description" content="The Dataverse Project is an open source software application to share, cite and archive data. Dataverse provides a robust infrastructure for data stewards to host and archive data, while offering researchers an easy way to share and get credit for their data." />

<link type="image/png" rel="icon" href="https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/images/favicondataverse.png.xhtml" />

<link type="image/png" rel="image_src" href="https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/images/dataverseproject.png.xhtml" />

<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.2/css/bootstrap.min.css" />

<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.2/css/bootstrap-theme.min.css" />

<link rel="stylesheet" href="https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/css/fontcustom.css" />

<link rel="stylesheet" href="https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/css/socicon.css" />

<link rel="stylesheet" href="https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/css/structure.css" />

<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.2/js/bootstrap.min.js"></script>

<style type="text/css">
@font-face {
font-family: "fontcustom";
src: url("https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/fontcustom/fontcustom_0cdeefae934823416d24b6c2132ac702.eot");
src: url("https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/fontcustom/fontcustom_0cdeefae934823416d24b6c2132ac702.eot?#iefix") format("embedded-opentype"),
url("https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/fontcustom/fontcustom_0cdeefae934823416d24b6c2132ac702.woff") format("woff"),
url("https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/fontcustom/fontcustom_0cdeefae934823416d24b6c2132ac702.ttf") format("truetype"),
url("https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/fontcustom/fontcustom_0cdeefae934823416d24b6c2132ac702.svg#fontcustom") format("svg");
font-weight: normal;
font-style: normal;
}

@font-face {
font-family: 'socicon';
src: url('https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/socicon-font/socicon-webfont.eot');
src: url('https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/socicon-font/socicon-webfont.eot?#iefix') format('embedded-opentype'),
url('https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/socicon-font/socicon-webfont.woff') format('woff'),
url('https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/socicon-font/socicon-webfont.woff2') format('woff2'),
url('https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/socicon-font/socicon-webfont.ttf') format('truetype'),
url('https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/socicon-font/socicon-webfont.svg#sociconregular') format('svg');
font-weight: normal;
font-style: normal;
text-transform: initial;
}

@media screen and (-webkit-min-device-pixel-ratio:0) {
@font-face {
font-family: "fontcustom";
src: url("https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/fontcustom/fontcustom_0cdeefae934823416d24b6c2132ac702.svg#fontcustom") format("svg");
}
}
</style>
</head>
<body>
<div id="dataverse-header-block">
<!-- Navbar Panel -->
<nav id="navbarFixed" class="navbar navbar-default navbar-fixed-top" role="navigation">
<div class="container">
<div class="navbar-header">
<span class="navbar-brand"><i id="icon-dataverse" class="icon-dataverse"></i> Dataverse</span>
</div>
<div class="collapse navbar-collapse" id="topNavBar">
<ul class="nav navbar-nav navbar-right">
<li>
&#160;
</li>
</ul>
</div>
</div>
</nav>

<!-- Header Panel -->
<div style="background:#F5F5F5;" class="container bg-muted" id="dataverseHeader">
<div class="dataverseHeaderBlock">
<div class="dataverseHeaderCell dataverseHeaderLogo">
<img alt="Harvard Dataverse" src="https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/doc/sphinx-guides/source/_static/installation/files/etc/maintenance/HarvardShield_RGB.png"/>
</div>
<div class="dataverseHeaderCell dataverseHeaderName">
<span class="dataverseHeaderDataverseName" style="color:#000000;">Harvard Dataverse</span>
</div>
<div class="dataverseHeaderCell dataverseHeaderTagline">
<span style="color:#a51c30;">A collaboration with Harvard Library, Harvard University IT, and IQSS</span>
</div>
</div>
</div>
</div>
<div class="container" id="content">
<div class="jumbotron" style="margin-top:20px;">
<h2><em>We apologize for the service interruption.</em></h2>
<p>The Harvard Dataverse is currently undergoing maintenance. At this time both the application and APIs are not able to be used. However, the datasets stored in the Harvard Dataverse are fine and not impacted by this maintenance.</p>
<p>If you have any comments, questions or concerns, please reach out to [email protected].</p>
</div>
</div>
<div class="container" id="footer">
<div class="row">
<div class="col-sm-8 small">
<p>Developed at the <a href="http://www.iq.harvard.edu/" title="Institute for Quantitative Social Science" target="_blank">Institute for Quantitative Social Science</a> &#160;|&#160; Dataverse Project on <a href="https://twitter.com/dataverseorg" target="_blank" title="Dataverse Project on Twitter"><span class="socicon socicon-twitter" title="Dataverse Project on Twitter"></span></a> &#160;|&#160; Code available at <a href="https://github.com/IQSS/dataverse" target="_blank" title="Dataverse On GitHub"><span class="socicon socicon-github" title="Dataverse On GitHub"></span></a>
</p>
<p>Copyright &#169; 2016, The President &#38; Fellows of Harvard College | <a href="http://best-practices.dataverse.org/harvard-policies/harvard-privacy-policy.html" target="_blank">Privacy Policy</a>
</p>
</div>
<div class="col-sm-4 text-right">
<div class="poweredbylogo">
<span>Powered by</span>
<a href="http://dataverse.org/" title="The Dataverse Project" target="_blank"><img src="https://cdn.rawgit.com/IQSS/dataverse/87cf58b5c3e8e01cf0f4891465df02334852eed3/src/main/webapp/resources/images/dataverseproject_logo.jpg" alt="The Dataverse Project" /></a>
</div>
</div>
</div>
</div>
</body>
</html>
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
module shibboleth 1.0;

require {
class file {open read};
class sock_file write;
class unix_stream_socket connectto;
type httpd_t;
type initrc_t;
type var_run_t;
type var_t;
}

allow httpd_t initrc_t:unix_stream_socket connectto;
allow httpd_t var_run_t:sock_file write;
allow httpd_t var_t:file {open read};
11 changes: 9 additions & 2 deletions doc/sphinx-guides/source/api/client-libraries.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Client Libraries
================

Currently there are client libraries for Python and R that can be used to develop against Dataverse APIs.
Currently there are client libraries for Python, R, and Java that can be used to develop against Dataverse APIs. We use the term "client library" on this page but "Dataverse SDK" (software development kit) is another way of describing these resources. They are designed to help developers express Dataverse concepts more easily in the languages listed below. For support on any of these client libraries, please consult each project's README.

Because Dataverse is a SWORD server, additional client libraries exist for Java, Ruby, and PHP per the :doc:`/api/sword` page.

Expand All @@ -10,11 +10,18 @@ Python

https://github.com/IQSS/dataverse-client-python is the offical Python package for Dataverse APIs.

`Robert Liebowitz <https://github.com/rliebz>`_ from the `Center for Open Science <http://centerforopenscience.org>`_ heads its development and the library is used to integrate the `Open Science Framework (OSF) <http://osf.io>`_ with Dataverse via an add-on which itself is open source and listed on the :doc:`/api/apps` page.
`Robert Liebowitz <https://github.com/rliebz>`_ created this library while at the `Center for Open Science (COS) <https://centerforopenscience.org>`_ and the COS uses it to integrate the `Open Science Framework (OSF) <https://osf.io>`_ with Dataverse via an add-on which itself is open source and listed on the :doc:`/api/apps` page.

R
-

https://github.com/IQSS/dataverse-client-r is the official R package for Dataverse APIs.

It was created by `Thomas Leeper <http://thomasleeper.com>`_ whose dataverse can be found at https://dataverse.harvard.edu/dataverse/leeper

Java
----

https://github.com/IQSS/dataverse-client-java is the offical Java library for Dataverse APIs.

`Richard Adams <http://www.researchspace.com/electronic-lab-notebook/about_us_team.html>`_ from `ResearchSpace <http://www.researchspace.com>`_ created and maintains this library.
47 changes: 41 additions & 6 deletions doc/sphinx-guides/source/api/native-api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,24 @@ Endpoints

Dataverses
~~~~~~~~~~~
Generates a new dataverse under ``$id``. Expects a json content describing the dataverse.
Generates a new dataverse under ``$id``. Expects a JSON content describing the dataverse, as in the example below.
If ``$id`` is omitted, a root dataverse is created. ``$id`` can either be a dataverse id (long) or a dataverse alias (more robust). ::

POST http://$SERVER/api/dataverses/$id?key=$apiKey

The following JSON example can be `downloaded <../_static/api/dataverse-complete.json>`_ and modified to create dataverses to suit your needs. The fields ``name``, ``alias``, and ``dataverseContacts`` are required. The controlled vocabulary for ``dataverseType`` is

- ``JOURNALS``
- ``LABORATORY``
- ``ORGANIZATIONS_INSTITUTIONS``
- ``RESEARCHERS``
- ``RESEARCH_GROUP``
- ``RESEARCH_PROJECTS``
- ``TEACHING_COURSES``
- ``UNCATEGORIZED``

.. literalinclude:: ../_static/api/dataverse-complete.json

View data about the dataverse identified by ``$id``. ``$id`` can be the id number of the dataverse, its alias, or the special value ``:root``. ::

GET http://$SERVER/api/dataverses/$id
Expand Down Expand Up @@ -70,12 +83,15 @@ Sets the metadata blocks of the dataverse. Makes the dataverse a metadatablock r

Get whether the dataverse is a metadata block root, or does it uses its parent blocks::

GET http://$SERVER/api/dataverses/$id/metadatablocks/:isRoot?key=$apiKey
GET http://$SERVER/api/dataverses/$id/metadatablocks/isRoot?key=$apiKey

Set whether the dataverse is a metadata block root, or does it uses its parent blocks. Possible
values are ``true`` and ``false`` (both are valid JSON expressions). ::

POST http://$SERVER/api/dataverses/$id/metadatablocks/:isRoot?key=$apiKey
PUT http://$SERVER/api/dataverses/$id/metadatablocks/isRoot?key=$apiKey

.. note:: Previous endpoints ``GET http://$SERVER/api/dataverses/$id/metadatablocks/:isRoot?key=$apiKey`` and ``POST http://$SERVER/api/dataverses/$id/metadatablocks/:isRoot?key=$apiKey`` are deprecated, but supported.


Create a new dataset in dataverse ``id``. The post data is a Json object, containing the dataset fields and an initial dataset version, under the field of ``"datasetVersion"``. The initial versions version number will be set to ``1.0``, and its state will be set to ``DRAFT`` regardless of the content of the json object. Example json can be found at ``data/dataset-create-new.json``. ::

Expand Down Expand Up @@ -131,8 +147,7 @@ Export the metadata of the current published version of a dataset in various for

GET http://$SERVER/api/datasets/export?exporter=ddi&persistentId=$persistentId

Note: Supported exporters (export formats) are ddi, oai_ddi, dcterms, oai_dc, and dataverse_json.

.. note:: Supported exporters (export formats) are ``ddi``, ``oai_ddi``, ``dcterms``, ``oai_dc``, and ``dataverse_json``.

Lists all the file metadata, for the given dataset and version::

Expand Down Expand Up @@ -268,6 +283,14 @@ Management of Shibboleth groups via API is documented in the :doc:`/installation
Info
~~~~

Get the Dataverse version. The response contains the version and build numbers::

GET http://$SERVER/api/info/version

Get the server name. This is useful when a Dataverse system is composed of multiple Java EE servers behind a load balancer::

GET http://$SERVER/api/info/server

For now, only the value for the ``:DatasetPublishPopupCustomText`` setting from the :doc:`/installation/config` section of the Installation Guide is exposed::

GET http://$SERVER/api/info/settings/:DatasetPublishPopupCustomText
Expand All @@ -286,7 +309,7 @@ Return data about the block whose ``identifier`` is passed. ``identifier`` can e

Admin
~~~~~~~~~~~~~~~~
This is the administrative part of the API. It is probably a good idea to block it before allowing public access to a Dataverse installation. Blocking can be done using settings. See the ``post-install-api-block.sh`` script in the ``scripts/api`` folder for details.
This is the administrative part of the API. For security reasons, it is absolutely essential that you block it before allowing public access to a Dataverse installation. Blocking can be done using settings. See the ``post-install-api-block.sh`` script in the ``scripts/api`` folder for details. See also "Blocking API Endpoints" under "Securing Your Installation" in the :doc:`/installation/config` section of the Installation Guide.

List all settings::

Expand Down Expand Up @@ -374,6 +397,18 @@ List all role assignments of a role assignee (i.e. a user or a group)::

Note that ``identifier`` can contain slashes (e.g. ``&ip/localhost-users``).

List permissions a user (based on API Token used) has on a dataverse or dataset::

GET http://$SERVER/api/admin/permissions/$identifier

The ``$identifier`` can be a dataverse alias or database id or a dataset persistent ID or database id.

List a role assignee (i.e. a user or a group)::

GET http://$SERVER/api/admin/assignee/$identifier

The ``$identifier`` should start with an ``@`` if it's a user. Groups start with ``&``. "Built in" users and groups start with ``:``. Private URL users start with ``#``.

IpGroups
^^^^^^^^

Expand Down
4 changes: 2 additions & 2 deletions doc/sphinx-guides/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,9 @@
# built documents.
#
# The short X.Y version.
version = '4.5.1'
version = '4.6'
# The full version, including alpha/beta/rc tags.
release = '4.5.1'
release = '4.6'

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
Expand Down
1 change: 1 addition & 0 deletions doc/sphinx-guides/source/developers/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,4 @@ Contents:
making-releases
tools
unf/index
selinux
Loading

0 comments on commit c913662

Please sign in to comment.